I'm creating a logic game based on Fox and Hounds game. The player plays the fox and AI plays the hounds. (as far as I can see) I managed to make the AI perfect, so it never loses. Leaving it as such would not be much fun for human players.
Now, I have to dumb-down the AI so human can win, but I'm not sure how. The current AI logic is based on pattern-matching - if I introduce random moves which make the board go out of pattern space the AI would most probably play dumb until the end of the game.
I'm also thinking about removing a set of patterns, so it would seem as AI does not know that "trick" but this way players could find a way to beat the computer using the same moves every time.
Any ideas how to dumb down the AI in such way that is does not go from "genius" to "completely dumb" in a single move?
We used MinMax as the AI algorithm for our game and we implemented the AI levels by setting different depth for each level
I ended up creating a couple of quasi-smart pattern plays (as if a 10 year old might play) so it is not completely dumb, and then I pick one or two of those at random before the game starts. This way the game is always beatable, but the player does not know how (i.e. he cannot use the same strategy to always win, he has to explore for weak spot first).
If your game is a zero-sum one, the MiniMax algorithm with an alpha-beta optimization is a good choice. You can create difficulty levels by making the search stop when the algorithm reaches a certain depth.
Since I'm not an expert game developer and my AI is actually too dumb at the moment, I'd make a sort of 'learning AI'. Say you keep all your regexes disabled and you enable them once the player uses them.
I think this is a zero-sum game, and you can use the MinMax algorithm to solve the game instead of pattern matching, in that way by controlling the search depth you can control the level of expertise of the agent.
On the other hand you can use the A* search to determine the best move for a given fox/hound. And choosing different heuristics you can control the effectiveness if the agent.
Related
I found the below text from here, saying that the result for minimax for games like tic-tac-toe and chess will always be a draw. I also saw minimax algorithms for unbeatable tic-tac-toe. But I don't quite understand the reason why minimax results in a draw. Is it because there is no guaranteed winning or losing move and thus the best possible option for both players is a draw?
a computer running a minimax algorithm without any sort of enhancements will discover that, if both it and its opponent play optimally, the game will end in a draw no matter where it starts, and thus have no clue as to which opening play is the "best." Even in more interesting win-or-lose games like chess, even if a computer could play out every possible game situation (a hopelessly impossible task), this information alone would still lead it to the conclusion that the best it can ever do is draw (which would in fact be true, if both players had absolutely perfect knowledge of all possible results of each move).
The information from the site you’ve linked is slightly incorrect.
We know from a brute-force exploration of the game that with perfect play tic-tac-toe will always end in a draw. That is, if both players play the game according to the best possible strategy, then the game ends in a draw. There’s a wonderful xkcd graphic that details how to play perfectly.
If you were to run a minimax search over the game all the way to the end, it isn’t necessarily the case that minimax won’t know what option to pick. Rather, minimax would select any move that leads to a forced draw, since it always picks a move that leads to the best possible result for the player. It’s “unbeatable” in the sense that a perfect minimax player will never lose and, if you play against it with a suboptimal strategy, it may be able to find a forced win and beat you.
As for chess - as of now (December 2021) no one knows whether chess ends in a draw with perfect play or whether one of the players has a forced win. We simply aren’t able to explore the game tree in that much depth. It’s entirely possible that white has a forced win, for example, in which case a minimax search given sufficient time and resources playing as white will always outplay you.
I have to implement card game similar to 3-5-8, but I have problem to select proper way for A.I. implementation. Is it possible game A.I. to be created with package like OpenRules business decision management system? I have looked at Drools also, but it seems too complicated.
Are expert systems, in general, proper for such A.I. development?
It depends on whether your solution employs heuristics or algorithms (or possibly a combination of both). For example, in chess you have complete knowledge of the game state and can plan turns in advance so a minimax algorithm allows you to find an "optimal" solution for any given search depth. A heuristic "good enough" solution can be used in situations where an optimal solution is not possible or difficult to determine (because the game state is not fully known, there is chance involved, or the number of player choices is too large to consider). For example, you can easily create basic heuristics for what to do in a blackjack or poker game based on the cards in your hand and the cards showing in other player hands. These simple heuristics could then be later expanded to include additional heuristics.
Here's a simple set of rules (i.e. heuristics) for winning a trick in a card game:
Rule FirstPlayHighTrump
When
You're playing the first card and
You have trump cards
Then
Play the highest trump card in your hand
Rule FirstPlayHighCard
When
You're playing the first card and
You don't have trump cards
Then
Play the highest card in your hand
Rule PlayHighCard
When
You're not playing the first or last card and
You have a card to win the trick
Then
Play the highest card in your hand that will win the trick
Rule PlayLowCard
When
You don't have a card to win the trick
Then
Play the lowest card in your hand
Rule LastPlayLowestCardToWin
When
You're playing the last card and
You have a card to win the trick
Then
Play the lowest card in your hand that will win the trick
These rules/heuristics are better than picking cards that follow suit at random, but since they don't involve planning for winning the next trick, they can be improved.
I need to implement an intelligent agent to play Abalone game, for this kind of game the best way to proceed seems a min-max strategy with alpha beta pruning.
I have already implemented a naive search algorithm that use min-max with pruning,
my problem is...
How to generate the nodes of the tree where perform the search?
I have no idea of the right way to do this, and how assign the weigh to each node.
For generating the tree nodes: You need to implement a method that returns a collection of all possible legal moves given the current board position and the player whose turn it is. All these moves will become children of the node representing the current board position. Repeat until memory is exhausted to generate the game tree ;) or rather until you reach a reasonable tree depth.
For alpha-beta search you also need an evaluation function which calculates the weight for each board position/node. You can do some research or think about such a function yourself, maybe considering the number of stones still on the board. However a bad evaluation function can seriously screw up your results, so take care and run a lot of tests.
If you have trouble coming up with a reasonable evaluation function, I recommend you take a look into Monte-Carlo techniques such as UCT.
http://en.wikipedia.org/wiki/Monte_Carlo_tree_search
These tackle the problem using a probabilistic approach and have some nice advantages over alpha-beta. Also they don't require an evaluation function so you could skip this step.
Good luck!
I have published two libraries for move generation in Abalone. You didn't mention the programming language used for your search implementation, but you can easily port the functions.
For C++, https://sourceforge.net/projects/abnet/
For Python, https://gitlab.com/peer.sommerlund/haliotis
For an evaluation function, distance between all your marbles, or distance to their gravity center (same thing), works nicely. Tino Werner used this with a twist for his program that won ICGA 2003.
For understanding distance when using hex coordinates, I can recommend Amit Patel's page: https://www.redblobgames.com/grids/hexagons/
I'm looking for guidance/advice on where to start to create a computer AI opponent for a match-3 game. This issue has been stumping me for years (literally), as I've not been able to figure it out. I've exhausted Google in finding this answer.
Sample match-3 games that have a computer opponent include: Puzzle Quest and Crystal Battle.
What programming methodologies are used in creating an AI opponent like this, and how can I apply it to Unity 2D scripting? Where/how can I start? I am mainly looking for a tutorial or something to get me started in the right direction. I realize this is not a quick an easy thing to do, but I would like to attempt it step by step so I can better understand things.
Thanks in advance!
There are two problems here:
Generating possible moves
Choosing the best move
If your board is reasonably small, you can simply brute-force both of them. For all positions in your grid check if you can move it up, down, left or right, and you have your move generator. (You should have checking for valid moves already implemented for a single-player version of the game).
Choosing the best move will be a bit more tricky, because you have to evaluate each move. Common way to do this is MiniMax method. General idea is that you build a tree of all possible moves in the next couple of turns and assign a score to each leaf. Then you reduce the tree so that parent node becomes max(leaves) if it is AIs turn to move, and min(leaves) if player moves. You end up with the score for your move at root.
Great resource for basic AI programming like this is Chess Programming Wiki (you won't need 90% of what is described there. Start with MiniMax and AlphaBeta algorithms).
On the other hand, for the simplest possible AI you can just pick a move at random, match-3 games are not the most demanding when it comes to planning your moves.
EDIT: As an afterthought, the following seems like a reasonable AI strategy for a match-3 game:
Assuming all the random gems added after each move cannot be matched in any way:
Pick a move that makes my opponent unable to move (has no child nodes).
If 1. is not possible, pick any move that guarantees me another move no matter which move my opponent picks (no child node is a leaf).
If 2. is not possible, pick random move.
How to use MinMax trees with Q-Learning?
I want to implement a Q-Learning connect four agent and heard that adding MinMax trees into it helps.
Q-learning is a Temporal difference learning algorithm. For every possible state (board), it learns the value of the available actions (moves). However, it is not suitable for use with Minimax, because the Minimax algorithm needs an evaluation function that returns the value of a position, not the value of an action at that position.
However, temporal difference methods can be used to learn such an evaluation function. Most notably, Gerald Tesauro used the TD(λ) ("TD lambda") algorithm to create TD-Gammon, a human-competitive Backgammon playing program. He wrote an article describing the approach, which you can find here.
TD(λ) was later extended to TDLeaf(λ), specifically to better deal with Minimax searches. TDLeaf(λ) has been used, for example, in the chess program KnightCap. You can read about TDLeaf in this paper.
Minimax allows you to look a number of moves into the future and play in a way to maximize your chances of scoring in that timespan. This is good for Connect-4, where a game can end almost at any moment and the number of moves available at each turn is not very large. Q-Learning would provide you with a value function to guide the Minimax search.
Littman has used minimax with Q learning. Hence proposed Minimix-Q learning algorithm in his famous and pioneering work Markov Games as a framework for multiagent reinforcement learning. His work is on zero-sum game in multiagent settings. Later Hu & Wellman extended his work to develop NashQ learning which you can find here.