Project Seesaw

Closure

2020-12-18T05:30:37+00:00

The Journey

Doing research on game balance started from my curiosity of hidden costs of balancing activities in game development. I wanted to know how balance techniques are used optimally to cut down costs and development time as well as create an intended experience for the players.

To better test balance techniques, I created a modified version of Rock Paper Scissors called Super Rock Paper Scissors which had competitive mechanics and allowed for well-thought strategies. Then, I made multiple AI agents which would play the game against each other in unique and exhaustive ways. I created a logger which would collect in-game data and dump into meaningful chunks of information. I then used excel files to automatically analyze the collected data and output meaningful metrics. I used the metrics to make meaningful changes to the game and iteratively test the game to figure out “when” a metric could be considered balanced. Lastly, I created this blog to track my findings and notes on the journey of how I balanced Super Rock Paper Scissors.

In the end, I realized that game balance begins in the earliest stages of design, and persists through all stages of development. Having a vision and goals for your game at all stages was crucial in order to validate balance changes. Despite the effectiveness of playtesting with human players, automating the game balance process through AI agents and data collection & analysis proved to be more cost effective and yielded meaningful game balance metrics comparable to playtesting with human players.

Builds

Github repo for Super Rock Paper Scissors: Click Here

Design Expo Presentation: Click Here

Download Links

Windows RPS_Windows.zip

Mac RPS_Mac.app.zip

Final Touch

2020-12-02T05:30:37+00:00

Machine Learning Trial

I read the chapters 6-7 of Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations by Yoav Shoham and Kevin Leyton-Brown to get a better idea of how I can implement empirical game theory and multi agent system that utilizes game theory.

I spent half of this weeks on getting the machine learning agents working; however, empirical game theory was a completely new field for me so I couldn’t get it to work with a 4-5 hour commitment. Although I was not able to implement machine learning this week, I do plan to finish what I started with a multi agent system that utilizes game theory.

Getting Ready For Design Expo

Expo Presentation

Created presentation slides and speech for the expo.

The slides can be accessed here

Small Tweaks

Number of Games to play

Added a field to the main menu that dictates how many games are going to be played.

Better Location For Log Files

Adjusted where log files are stored in, for accessibility.

Game Settings

Tweaked costs and scoring to be optimal for a 20-score game.

Tie Breaking

Adjusted the tie breaking logic.

UI Adjustments

Made it so that the UI doesn’t shift into play-field in common screen ratios such as 16:9.
Added Title

Statistics Excel Files

Adjusted data analysis files for better accessibility.

Builds

Super Rock Paper Scissors can be downloaded here

Windows RPS_Windows.zip

Mac RPS_Mac.app.zip

Weekly Time Spent on Project SeeSaw

Day	Hours Spent
Thu	0
Fri	0
Sat	0
Sun	0
Mon	1:00
Tue	4:00
Wed	4:00
TOTAL	9:00

Tweaking

2020-11-18T05:30:37+00:00

Unity Shenanigans Vol. 3

Unity decided to stop working once again. Despite all my troubleshooting efforst, nothing seemed to get Unity working again until I reinstalled Unity and Unity Hub.

Readme.md

I created a readme file for the project’s github repo guiding people who might be interested in running and modifying the repo for their own research. The readme can be accessed here

Tie Breaking

To solve the indefinite stalemate loops, I decided to go with the tie method I discussed last week.

To recap:

The Tie Breaker Trigger prevents indefinite stalemate chains by detecing when a stalemate chain of X rounds occurs, and applying a tie breaking scenario to enforce a round winner on the third consecutive stalemate round. This limits max game length of a single game to

MAXGAMELENGTH = (ScoreToWin + (ScoreToWin - 1)) * ((MaxStalemateChainLength - 1) + 1)

The tie breaker scenario could look compare total stats and/or coins of players to determine a winner. While this shrinks the possibility of having a truly random round to nearly zero, it does not completely eliminate the possibility of having indefinite stalemates. Since this scenario is rare enough, the round winner can be decided at random when compared stats of both players are equal. The introduction of randomness here is much better than the random nature of the Max Rounds approach with the example Sudden Death Scenario I talked about above.

Tie Breaker Logic

The Tie Breaker tries to break the tie as much as possible with the following steps:

It first considers the total stats. The player with the highest stats win
If both players have equal total stats, then the player with most coins wins
If both players have equal coins, then the player with most total upgrades and cards wins
If both players have equal total upgrades and cards, then the player with the higher score wins
If both players have equal score and there is only 1 human player, then the human player wins
If there are no human players or both players are human, then a random player wins

Tie Breaker Code

public WinnerData StalemateStreakBreaker(WeaponController LWeapon, WeaponController RWeapon)
{
    float LTotWeaponStats = playerControllerL.GetTotalWeaponStats();
    float RTotWeaponStats = playerControllerR.GetTotalWeaponStats();
    //The player with highest total stats win
    if (LTotWeaponStats > RTotWeaponStats)
    {
        return new WinnerData(Player.L, -1, LWeapon.weaponType, RWeapon.weaponType);
    }
    else if (RTotWeaponStats > LTotWeaponStats)
    {
        return new WinnerData(Player.R, -1, LWeapon.weaponType, RWeapon.weaponType);
    }
    else
    {
        //The player with most coins win
        if (playerControllerL.coins > playerControllerR.coins)
        {
            return new WinnerData(Player.L, -1, LWeapon.weaponType, RWeapon.weaponType);
        }
        else if (playerControllerR.coins > playerControllerL.coins)
        {
            return new WinnerData(Player.R, -1, LWeapon.weaponType, RWeapon.weaponType);
        }
        else
        {
            //The player with most upgrades and cards win
            int totLCardUpgrades = playerControllerL.GetTotalCards() + playerControllerL.GetTotalUpgrades();
            int totRCardUpgrades = playerControllerR.GetTotalCards() + playerControllerR.GetTotalUpgrades();
            if (totLCardUpgrades > totRCardUpgrades)
            {
                return new WinnerData(Player.L, -1, LWeapon.weaponType, RWeapon.weaponType);
            }
            else if (totRCardUpgrades > totLCardUpgrades)
            {
                return new WinnerData(Player.R, -1, LWeapon.weaponType, RWeapon.weaponType);
            }
            else
            {
                //The player with higher score wins
                if (playerControllerL.score > playerControllerR.score)
                {
                    return new WinnerData(Player.L, -1, LWeapon.weaponType, RWeapon.weaponType);
                } 
                else if (playerControllerR.score > playerControllerL.score)
                {
                    return new WinnerData(Player.R, -1, LWeapon.weaponType, RWeapon.weaponType);
                } else
                {
                    //Human player wins
                    if (GlobalVars.instance.LType == PlayerType.Human ^ GlobalVars.instance.RType == PlayerType.Human)
                    {
                        if (GlobalVars.instance.LType == PlayerType.Human)
                        {
                            return new WinnerData(Player.L, -1, LWeapon.weaponType, RWeapon.weaponType);
                        }
                        else
                        {
                            return new WinnerData(Player.R, -1, LWeapon.weaponType, RWeapon.weaponType);
                        }
                    }
                    else
                    {
                        //Random player wins
                        if (randObj.Next(0, 2) == 0)
                        {
                            return new WinnerData(Player.L, -1, LWeapon.weaponType, RWeapon.weaponType);
                        } else
                        {
                            return new WinnerData(Player.R, -1, LWeapon.weaponType, RWeapon.weaponType);
                        }
                    }
                }
            }
        }
    }
}

Tie Breaker Performance

The game length significantly decreased when Simp_ScissorLover plays against itself. Remember that ScissorLover only upgrades scissor, and chooses scissor 80% of the time.

Pre Tie Breaking	Post Tie Breaking

Previously Impossible Matchup

The Simp_ScissorLover2 AI chooses scissor 100% of the time, which previously meant that when Simp_ScissorLover2 plays against itself, the game would get stuck in an indefinite stalemate loop.

With the tie breaker logic implemented, this matchup now results in valid games.

As it can be seen from the graph, the games now end with a definite winner. The competitiveness score goes now as Score To Win increases because after the first tie breaker results in a random winner, the game snowballs to the winner breaker’s favor due to how Simp_ScissorLover2 works.

Interview with Sindhu Kutty

Talked to University of Michigan Machine Learning professor Sindhu Kutty about how I can utilize ML and deep learning to create an adaptive AI.

She suggested that I look into empirical game theory and using a multi agent system that utilizes game theory.

These are the readings she suggested I look into:

Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations

Methods for Empirical Game-Theoretic Analysis

Essentials of Game Theory

Algorithmic Game Theory

Mastering the game of Go without human knowledge

Links

Github repo for Super Rock Paper Scissors: Click Here

Weekly Time Spent on Project SeeSaw

Day	Hours Spent
Thu	0
Fri	0
Sat	2:00
Sun	2:00
Mon	0:00
Tue	4:00
Wed	4:00
TOTAL	12:00

Machine Learning and Research

2020-11-11T05:30:37+00:00

Machine Learning For AI Players Research

This week, I spent most my time researching about machine learning techniques, and writing the infrastructure for testing couple of these machine learning tools.

Unsupervised vs Supervised

Research

Unsupervised vs Supervised ML

Big Data Bioinformatics

Conclusion

First question to tackle was what kind of machine learning would best fit a Super Rock Paper Scissors AI player.

Since the AI players will be able to accumulate information about their opponent’s previous actions, the supervised approach seems more fitting.

Classification vs Regression

Research

Classification vs Regression ML

Conclusion

We want the machine learning model to predict what the opponent is gonna do this round. The opponent’s actions can be represented as a finite sized set of classifications. For example, the opponent can upgrade Scissor, play Defense Increase Card, and pick Scissor which can be represented by the classification (US,DIC,PS).

In conclusion, it seems like the Supervised Classification Learning fits the bill the best.

Feature Space

Research

First I started looking into Convolutional Neural Networks

Predicting Professional Players’ Chess Moves with Deep Learning

Predicting Moves in Chess using Convolutional Neural Networks

Then I realized, the snapshots of my game state would depend on each other due to the importance of ordering and timing of the game state snapshots. I then started looking into Recurrent Neural Nets

Convolutional Neural Network for time-dependent features

Recurrent Convolutional Neural Networks for Scene Labeling

Attention, please: forget about Recurrent Neural Networks

Deep Learning, CNN - RNN - Pytorch

Conclusion

The dimensionality of the feature space and its time-dependence is a more intricate question than I anticipated. It seems like using an RNN model will be the best way to move forward, but before I pick a machine learning model I want to discuss it with a Machine Learning Professor.

A potential CNN solution I’ve came up with follows the model below.

# Insired from University of Michigan EECS 445 Project 2
class CNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = torch.nn.Conv2d(3,16,(5,5), stride=(2,2), padding=self.get_same_padding(5))
        self.conv2 = torch.nn.Conv2d(16,64,(5,5), stride=(2,2), padding=self.get_same_padding(5))
        self.conv3 = torch.nn.Conv2d(64,32,(5,5), stride=(2,2), padding=self.get_same_padding(5))
        self.fc1 = torch.nn.Linear(512,64)
        self.fc2 = torch.nn.Linear(64,32)
        self.fc3 = torch.nn.Linear(32,3)
        self.init_weights()

    def get_same_padding(self, filter_dim):
        return ( int((filter_dim - 1) / 2), int(( filter_dim - 1) / 2))

    def init_weights(self):
        for conv in [self.conv1, self.conv2, self.conv3]:
            C_in = conv.weight.size(1)
            nn.init.normal_(conv.weight, 0.0, 1 / sqrt(5*5*C_in))
            nn.init.constant_(conv.bias, 0.0)
        for fc in [self.fc1, self.fc2, self.fc3]:
            C_in = fc.weight.size(1)
            nn.init.normal_(fc.weight, 0.0, 1 / sqrt(C_in))
            nn.init.constant_(fc.bias, 0.0)

    def forward(self, x):
        N, C, H, W = x.shape
        print(x.shape)
        x1 = F.relu(self.conv1(x))
        x2 = F.relu(self.conv2(x1))
        x3 = F.relu(self.conv3(x2))
        x3 = torch.flatten(x3, 1)
        x4 = F.relu(self.fc1(x3))
        x5 = F.relu(self.fc2(x4))
        x6 = self.fc3(x5)

        return x6

Potential Solutions to Fixing Indefinite Stalemates

Why is Indefinite Stalemates a serious issue

As I envisions Super Rock Paper Scissors to be a proper competitive game that can be played in a tournament structure, the game length is supposed to be reasonably predictable. It should not be possible to extend one game indefinitely, as that would make the tournament structure vulnerable to delaying schemes. In the case of unintentional stalemate chains, the game should have a predictable length to allow players to predict and plan a reasonable timeframe for playing Super Rock Paper Scissors.

To show how long stalemates can exten the gamelength, I made Simple Scissor Lover AI which picks Scissor 80% of the time and the other weapons 20% of the time play against itself.

The image above shows the competitiveness score across the board are around double the competitiveness score of a regular game of Super Rock Paper Scissors (1.3 to 1.8). The graph shows, as the game gets longer, stalemates become more unlikely as the players gain resources to upgrade and buy cards which essentially break the ties when both players pick the same weapon. When the game is long enough; however, (Score To Win set to 100) we can see that once max upgrades is reached, the stalemate scenarios start to become likely again.

Fixing indefinite stalemates was a harder issue than I thought it would be. This is mainly because I did not like the idea of allowing games that end in a stalemate, since I think games with definite winners are more flexible in a competitive environment.

Fatigue System

Fatigue system tries to tackle the indefinite stalemates issue by punishing players for sticking to playing the same weapon or weapon pattern over and over again so that the players are demotivated to be predictable. It does this by debuffing stats of each weapon depending on how often they were used in the past X rounds. The premise that the players would be less likely to be caught in stalemate loops may be true but the fatigue system does not entirely eliminate the possibility of having unintentional stalemate chains, let alone intenional stalemates. One other problem with the fatigue system is that I think it is an intricate and tedious new mechanic only clutters the gameplay with uneccesary micromanagement.

Max Rounds

One idea is to limit max possible rounds that can be played in a single game. While limiting maximum possible rounds fix the issue of indefinite stalemate, it introduced couple new issues. First of all, it does not provide a solution to deciding a winner when the max possible rounds is reached. Second of all, enabling games to be concluded before at least Score To Win many rounds have been played makes it difficult to predict and control how much money and resources will be available to both players in an average game of Super Rock Paper Scissors.

Max Rounds With Sudden Death

One improvement to the Max Rounds idea to implement a sudden death scenario in case of a stalemate. One idea for the sudden death scenario is to make both players play a best of 3 game of traditional Rock Paper Scissors (no upgrades and cards) against random AI players. Whoever player reaches First player to win 2 rounds against the random AI win the game.

While Max Rounds With Sudden Death fixes the potential of having indefinite stalemates, it still does make balancing and predicting the amount of money and resources in circulation harder from a game balance standpoint. Max Rounds With the suggested sudden deatch scenario also introduces randomness as a deciding factor of the winner of a stalemate game.

Time Limit (With or Without Sudden Death)

Time limit is similar to Max Rounds as they share the same flaws and benefits. One characteristic of time limit that stands out over max rounds however is it makes game length more predictable which may allow for a better competitive tournament environment. Time limit; however, can be abused by players by taking as long as possible while winning. This can be partially mitigated by implementing a time limit for each round separate from the game time limit.

Tie Breaker Trigger

MAXGAMELENGTH = (ScoreToWin + (ScoreToWin - 1)) * ((MaxStalemateChainLength - 1) + 1)

Links

Github repo for Super Rock Paper Scissors: Click Here

Current Project Code Structure Of Super Rock Paper Scissors

Weekly Time Spent on Project SeeSaw

Day	Hours Spent
Thu	0:00
Fri	0:00
Sat	2:00
Sun	0:00
Mon	4:00
Tue	4:00
Wed	2:00
TOTAL	12:00

Data Collection 2

2020-11-04T05:30:37+00:00

Long Term Viability Of Cards

Random Upgrade vs Random Card	Random vs Random Card	Random vs Random Upgrade

These 2 matchups show that hybrid use and card focus strategies are inferior to upgrade focus strategy in a randomized environment. Although, in theory card strategy can perform better if the player manages to pull of many well-timed cards in one game, I still wanted cards to have a better cost-benefit ratio in the long term by decreasing card costs and observing the results.

Experimentation with Card Usage Ratio and Card Price

The table below shows the 200-game results when card usage ratio and card price change.

RandAI vs RandUpgrade	Expensive Cards	Cheap Cards
RandAI High Card Buy Ratio
RandAI Low Card Buy Ratio

The table shows that when the card prices are halved, the hybrid strategy beats the upgrade-only strategy around when score to win within 5-50 range

Adaptive AI: Mid Tracker vs Random and Mid Tracker vs Simple Scissor Lover

First to both test the MidTracker AI and to prove following a simple and boring strategy is not sufficient to win in Super Rock Paper Scissors, I made Mid Tracker play against Simple Scissor Lover.

Proving Adaptive Strategy Beats Simple 1-Weapon Strategy

As seen from the results, Simple Scissor Lover stands no change against Mid Tracker

Tracker vs Random AI Problem

Although Mid Tracker’s ability to form an opinion based on past actions by their opponent, its simple “choose the strong type of whatever I believe my opponent will choose” does not help enough against a random strategy.

Better Tracker

The improved Mid Tracker2 has a weapon bias. It calculates probabilities of the opponent using a specific weapon similar to Mid Tracker, but unlike Mid Tracker the improved Mid Tracker 2 has a default prediction value for its biased weapon choice’s strong type. For example let’s assume Mid Tracker 2 keeps track of the past 3 weapons its opponent played which in order are: Rock, Scissor, Paper. This would mean that the prediction probabilities for Rock Paper and Scissor are, 0.33, 0.33, and 0.33. However, if Mid Tracker 2 has a bias for Paper with a random bias value of 0.37, then the tie would be broken and Mid Tracker 2 would choose Paper’s strong type which is rock.

The Random AI gradually starts winning because Tracker AI relies too much on cards and not much on upgrades, so the switch at the 100 score to win mark can be ignored as the card/upgrade ratio wasn’t well controlled in this experiment. This is something I am plan to fix next week.

Next Week

Next week I want to look into game length. In theory, games can last forever if both players are following an identical deterministic strategy. I want to explore the implications of having a constraint that disallows such scenarios.

Links

Github repo for Super Rock Paper Scissors: Click Here

Current Project Code Structure Of Super Rock Paper Scissors

Weekly Time Spent on Project SeeSaw

Day	Hours Spent
Thu	0
Fri	0
Sat	2:00
Sun	2:00
Mon	0:00
Tue	0:00
Wed	4:00
TOTAL	8:00

Data Collection

2020-10-28T05:30:37+00:00

Excel For Data Analysis

I decided to use excel to summarize and analyze the logs. I created an excel file which creates a summary table and graphs for generated csv logs

Example_Excel_Summary_Sheet

Example_Excel_Graph_Sheet

Bug Discovery

Finally having the Game, AI Bots, and logging working properly, I was ready to start collecting data. Before I did anything fancy, I wanted to prove that a random AI can be beaten by some simple strategy, hence proving that I succeeded at decoupling chance factor from winning a game of Rock Paper Scissors.

On paper, it seemed like SimpleScissorLover should be able to take down a random strategy since focusing on upgrading and buffing 1 weapon usually means you win 2 of the 3 possible matchups given enough rounds to play.

However, when I actually ran the tests, I couldn’t make sense of the results.

As can be seen by the results, random strategy was winning against all the revisions of SimpleScissorLover AI. I calculated the theoretical probability of random strategy winning, and the math did not align with the results. I suspected there might be a bug in the winner determination code. Debugging the code revealed there, in fact, was a bug.

The code above handles determining the winner when both players pick the same weapon. As can be seen in the code, the Left player had a huge advantage since the second else clause that is suppose to check if player R won the round had it a faulty conditional expression that would never let player R win.

Proving Starting Conditions Are Fair

To prove that the starting conditions are fair for both players, I made Random AI play against itself 200 games with score to win set to 20.

These were the results

As can be seen from the results, the difference between the win ratios is small enough to be statistically insignificant over 200 games of Super Rock Paper Scissors. This hints that the starting conditions for both left and right players are fair and symmetrical.

Proving Random Strategy Can Be Beaten

Once I fixed the bug, I moved on to proving that the random strategy was not optimal. I think this is an important first step, because proving random strategy to be subpar to other strategies hints that the game can have interesting competitive decisions, which from a game balance standpoint is a neccessity for a 1v1 game to have.

Before I show you the data though, I want to introduce the competetiveness score I’ve come up with to better interpret the data.

Competitiveness Metric

I’ve come up with a competitiveness metric to better make sense of the data.

It’s calculated as such:

Sum up round wins for each round outcome (player L wins, player R wins, stalemate) across all logged games
Find the average round wins per game for each round outcome
Divide the averages by the score needed to win
Add up all these averages for each round outcome
The total score represents game length (competitiveness) and each individual averages for round outcomes represent individual performace (performance of players and stalemate).

Simp_Random Vs Simp_ScissorLover2

I’ve made these 2 AI play against each other 5 games with differing score to win values (1,5,20,50,100). Score to win represents the score which a player needs to reach to win the game, score is earned by winning rounds.

Win Ratio

As the score to win increases, random strategy becomes less viable. This is because during the early stages of the game, no player has the resources to utilize cards and upgrades yet, thus the early-game of Super Rock Paper Scissors resembles the original Rock Paper Scissors where a random strategy is viable.

Competitiveness

The competitiveness score follows a similar trend where ScissorLover’s performance increases and Random’s performance decreases as score to win increases. However, what is interesting is seing the competitiveness score increase while the winning strategy is switching from Random to ScissorLover. The competitiveness is highest when score to win is set to 5.

Observations and Conclusion

This experiment not only proves that the game potentially consists of interesting competitive decisions, but also gives insight to what may be an optimal value for score to win where random strategy is deemed unviable.

Cards Vs Upgrades

To see how impactful cards and upgrades are, I made Random AI play against altered versions of itself

Versions of Random AI and their behavior

Random AI (hybrid):
- Tries to buy a card 66% of the time.
- Tries to upgrade 50% of the time.
- Chooses each weapon with 33.3% chance
RandomUpgrade AI:
- *Never tries to buy a card
- Tries to buy an upgrade 33% of the time if the player has less than 5 coins
- Tries to buy an upgrade 100% of the time (prefers attack upgrade) if player has 5 or more coins
- Chooses each weapon with 33.3% chance
RandomCard AI:
- Tries to buy a card every round and use it
- Never tries to upgrade
- Chooses each weapon with 33.3% chance

Random AI vs RandomUpgrade AI

In the early game, focusing on only upgrades seems to be the inferior strategy. However, when the score to win is about 10-20, the upgrade strategy starts beating hybrid strategy.

Random AI vs RandomCard AI

In the early game, focusing on only cards seems to be the superior strategy. However, when the score to win is about 5-20, the hybrid strategy starts beating card strategy.

Comparison

Random vs Random Upgrade	Random vs Random Card

The late-game benefit of upgrades can be seen taking effect in both Win Ratio graphs. Upgrades is an investment, so given enough time, they change the tide of the game. In comparison, the Random Vs Random Card Win Ratio graph suggests that cards are good for short term benefit when the number of rounds per game is fewer.

The overall high competitive scores suggest that, even though there are superior and inferior strategies, in short-cycles all strategies can be competitive and win.

In conclusion, as the average number of rounds per game incrase, the card strategy and hybrid strategies start losing. Hybrid strategy is losing against upgrade strategy hints that the cost-benefit of using cards is perhaps imbalanced where the cost does not justify the potential benefit. This is something I will address next week.

Next week I will make Random Upgrade AI play against Random Card AI to double-check this hypothesis.

More AI and Using Python in Unity to Enable Creating Machine Learning Models

I am currently working on getting these AI ready for data collection:

Greedy AI: The previous implementation of greedy AI had many setbacks which needed fixing
Mid Tracker AI: An AI that tracks the opponents last N played weapons and predicts what the opponent is going to play this round and acts accordingly
Random AI with Card or Upgrade focus: These AI were created out of neccessity to measure the usefulness of cards and upgrades in a vacuum.
Base MLAI and its implementations: MLAI stands for Machine Learning AI which every now and then trains a model on their opponent’s previous picks and tries to predict their next weapon

Base MLAI

The implementation for base MLAI requires the use of Python as Python has much better support for machine learning endeavors. I had to look into how Python code can be run through Csharp and Unity. I found couple of different approaches including a client-server approach which seemed super neat but lacked the comprehensive documentation I needed. I decided to go with a simple solution which uses the Python interpreter in the current execution environment.

The code for calling Python code:

Links

Github repo for Super Rock Paper Scissors: Click Here

Webbuild: Click Here

Citations & Resources

1) https://github.com/gilcoder/AI4U/tree/master/doc

2) https://docs.unity3d.com/Packages/com.unity.scripting.python@2.0/manual/index.html

3) https://pythonpedia.com/en/tutorial/10759/call-python-from-csharp

4) https://www.softwarepronto.com/2016/09/calling-python-from-netc.html

5) https://support.microsoft.com/en-us/office/use-autosum-to-sum-numbers-543941e7-e783-44ef-8317-7d1bb85fe706#:~:text=If%20you%20need%20to%20sum,Here’s%20an%20example.

6) https://www.excel-easy.com/examples/average.html

7) https://exceljet.net/formula/sum-if-cells-contain-specific-text

8) https://smallbusiness.chron.com/add-cells-across-multiple-spreadsheets-44275.html

Current Project Code Structure Of Super Rock Paper Scissors

Weekly Time Spent on Project SeeSaw

Day	Hours Spent
Thu	0
Fri	0
Sat	0
Sun	4:30
Mon	5:15
Tue	4:15
Wed	2:00
TOTAL	16:00

Logging

2020-10-21T14:30:37+00:00

Logging

This week I was able to implement a solid framework for logging and automated testing.

When I was initially creating the automated logging framework, I immediately started finding bugs and balance issues. They ranged from minor visual errors, to game crashing bugs. So in some sense, implementing the testing framework itself was the first integration test Super Rock Paper Scissors went through.

Once all bugs got fixed, I put it to test by making simple AI play the game. Even with simple AI, the logs revelead the first plausible game balance change.

First Game Balance Change Discovered by Logging and AI

Observation

AI that focuses on upgrading Defense and/or using Defense-buff cards tend to lose more often than attack-centric AI.

Deduction

Defense buffs never give you a winning edge over your opponent. Defense is only useful when you lose the round and you want to minimize your opponent’s earnings. Defense buffs are either too expensive or defense is a redundant attribute.

Solution To give defense attribue a purpose, I decided to make defense have an impact on picking the round winner.

Balance Changes

When both players pick the same weapon, their weapon’s Attack and Defense attributes are summed up. Whichever player has the greatest sum win the round. Coins are calculated as winner's sum - loser's sum + 1

Conclusion

Since defense is also cheaper to upgrade, this means it’s crucial to have high defense in Weapon X vs Weapon X rounds to both win the round and earn large amount of coins that you would not be able to win in any other X vs Y round.

So now defense has a role in deciding the round winner, as well as acting as a trump card to win a lot of coins in one round.

I am amazed that even in the early stages of creating AI and Logging tools to balance the game, the tools are already helping me find out glaring issues with the game design.

First Bug Discovered By Simple AI

Expectation

When a card is used, its effect is active only for the next round.

Observation

When greedy attacker uses a card, its effect sometimes lasts for multiple rounds

Solution

Fixed the code where the child card class was wrongly calling its parent’s disable function.

Logger

Logger writes to 2 files:

A text file keeping track of events and actions vs time
A CSV file keeping track of reports generated at the end of a game consisting of over 50+ datapoints of information.

Example Text Log

Example CSV Data Log (1 Game, Transposed 90 Degrees)

Datapoint Number	Datapoint Name	Value
0	LPlayerType	Human
1	RPlayerType	Simp_ScissorLover
2	GameWinner	R
3	TotalRounds	3
4	NumLRoundWins	1
5	NumRRoundWins	2
6	NumStalemates	0
7	LLongestWinStreak	1
8	RLongestWinStreak	2
9	LTotalCards	2
10	RTotalCards	0
11	LTotalUpgrades	1
12	RTotalUpgrades	1
13	LWeaponUsage_Scissor	1
14	LWeaponWins_Scissor	1
15	LWeaponUsage_Paper	2
16	LWeaponWins_Paper	0
17	LWeaponUsage_Rock	0
18	LWeaponWins_Rock	0
19	RWeaponUsage_Scissor	3
20	RWeaponWins_Scissor	2
21	RWeaponUsage_Paper	0
22	RWeaponWins_Paper	0
23	RWeaponUsage_Rock	0
24	RWeaponWins_Rock	0
25	LWeaponUsage%_Scissor	0.333
26	LWeaponWins%_Scissor	1
27	LWeaponUsage%_Paper	0.667
28	LWeaponWins%_Paper	0
29	LWeaponUsage%_Rock	0
30	LWeaponWins%_Rock	0
31	RWeapoNUsage%_Scissor	1
32	RWeaponWins%_Scissor	0.667
33	RWeaponUsage%_Paper	0
34	RWeaponWins%_Paper	0
35	RWeaponUsage%_Rock	0
36	RWeaponWins%_Rock	0
37	LUpgrade_Scissor_Attack	0
38	LUpgrade_Scissor_Defense	0
39	LUpgrade_Paper_Attack	0
40	LUpgrade_Paper_Defense	1
41	LUpgrade_Rock_Attack	0
42	LUpgrade_Rock_Defense	0
43	RUpgrade_Scissor_Attack	1
44	RUpgrade_Scissor_Defense	0
45	RUpgrade_Paper_Attack	0
46	RUpgrade_Paper_Defense	0
47	RUpgrade_Rock_Attack	0
48	RUpgrade_Rock_Defense	0
49	LSelfAttackIncreaseAdditiveCurrent1	1
50	LSelfDefenseIncreaseAdditiveCurrent1	1
51	RSelfAttackIncreaseAdditiveCurrent1	0
52	RSelfDefenseIncreaseAdditiveCurrent1	0

Example log files with 300 games of data

CSV_LOG

TXT_LOG

Unity Based Research

This week’s work required me do research on couple of the topics listed below:

Reading/Writing To Files in Unity

Using External CSharp Libraries Within Unity

Using Different Target Framework Version With Unity C project

The hour long hunt to find a 1-liner bug

The code below somehow does not load the menu scene and does not throw a warning or error

When the menu scene does not load, the game gets stuck in an infinite game loop.

Turns out the scene was not loading because I forgot to unsubsribe one of the subscriptions. The floating subscription started a chain of events which eventually stopped the scene from changing.

Links

Github repo for Super Rock Paper Scissors: Click Here

Webbuild: Click Here

Current Project Code Structure Of Super Rock Paper Scissors

Weekly Time Spent on Project SeeSaw

Day	Hours Spent
Thu	0
Fri	0
Sat	1:15
Sun	5:30
Mon	3:45
Tue	2:00
Wed	0
TOTAL	14:00

AI Players

2020-10-14T14:30:37+00:00

AI Players For Balancing

Started designing AI that would play the game in unique ways. There are currently 4 available CPU players: Simple Scissor Lover, Greedy Attacker, Greedy Defender, and Greedy Mixed

I plan to make these AI play against each other while a logger script keeps track of game length, popular weapons, popular upgrades, popular cards, number of stalemates, win percentage per strategy, card and upgrade usefulness (does buying cards or upgrading weapons increase your chances of winning), defense vs attack based strategy win ratios and potentially more datapoints.

Base AI

All AI scripts extend the base AI class which has virtual functions for each phase (Buy, Action, Attack).

public abstract class BaseAI : MonoBehaviour
{
    protected Player player;
    protected PlayerController pc;
    protected float waitDurBetweenPhases = 0;

    protected static System.Random randObj = new System.Random();

    public bool active = false;

    public Subscription<CurrentPlayerChanged> CurrentPlayerChangedSubscription;
    public Subscription<PlayerWonRound> PlayerWonRoundSubscription;

    void Awake()
    {
        CurrentPlayerChangedSubscription = _EventBus.Subscribe<CurrentPlayerChanged>(_OnCurrentPlayerChanged);
        PlayerWonRoundSubscription = _EventBus.Subscribe<PlayerWonRound>(_OnPlayerWonRound);
    }

    public static Type GetAIType(PlayerType type)
    {
        switch (type)
        {
            case PlayerType.Simp_ScissorLover:
                return typeof(Simp_ScissorLover);
            case PlayerType.Mid_GreedyAttacker:
                return typeof(Mid_GreedyAttacker);
            case PlayerType.Mid_GreedyDefender:
                return typeof(Mid_GreedyDefender);
            case PlayerType.Mid_GreedyMixed:
                return typeof(Mid_GreedyMixed);
        }
        //Human
        return null;
    }

    public void InitializeBase(Player _player)
    {
        pc = GameController.instance.GetPlayer(_player);
        player = _player;
        active = true;
        Initialize();
    }

    void _OnCurrentPlayerChanged(CurrentPlayerChanged e)
    {
        if (!active)
        {
            return;
        }

        if (e.newCurr == player)
        {
            StartCoroutine(DoTurn());
        }
    }

    void _OnPlayerWonRound(PlayerWonRound e)
    {
        if (!active)
        {
            return;
        }

        if (e.player_t == player)
        {
            PostAttackPhase(true);
        }
        else
        {
            PostAttackPhase(false);
        }
    }

    protected IEnumerator DoTurn()
    {
        yield return 0;

        //=== BUY PHASE ===
        BuyPhase();
        if (waitDurBetweenPhases != 0)
        {
            yield return new WaitForSeconds(waitDurBetweenPhases);
        }

        //=== ACTION PHASE ===
        ActionPhase();
        if (waitDurBetweenPhases != 0)
        {
            yield return new WaitForSeconds(waitDurBetweenPhases);
        }

        //=== ATTACK PHASE ===
        AttackPhase();
        if (waitDurBetweenPhases != 0)
        {
            yield return new WaitForSeconds(waitDurBetweenPhases);
        }

        yield break;
    }

    protected virtual void Initialize()
    {
        return;
    }

    protected virtual void BuyPhase()
    {
        _EventBus.Publish<EndTurnPhase>(new EndTurnPhase(pc));
    }

    protected virtual void ActionPhase()
    {
        _EventBus.Publish<EndTurnPhase>(new EndTurnPhase(pc));
    }

    protected virtual void AttackPhase()
    {
        _EventBus.Publish<EndTurnPhase>(new EndTurnPhase(pc));
    }

    protected virtual void PostAttackPhase(bool isWinner)
    {
        return;
    }
}

Simple Scissor Lover

This AI only upgrades its scissor weapon and has an 80% chance of playing scissor in the attack phase.

public class Simp_ScissorLover : BaseAI
{
    protected override void Initialize()
    {
    }

    protected override void BuyPhase()
    {
        //Buys Nothing
        _EventBus.Publish<EndTurnPhase>(new EndTurnPhase(pc));
    }

    protected override void ActionPhase()
    {
        //If possible, always upgrades scissor's attack
        if (pc.CanUpgradeWeapon(WeaponType.Scissor, WeaponAttribute.Attack))
        {
            _EventBus.Publish<WeaponUpgraded>(new WeaponUpgraded(pc, WeaponType.Scissor, WeaponAttribute.Attack));
        }
    }

    protected override void AttackPhase()
    {
        //Has a 10% change of choosing rock and 10% chance of choosing paper, otherwise chooses scissor 
        int rand = randObj.Next(0, 100);
        if (rand < 10)
        {
            _EventBus.Publish<AttackWeaponPicked>(new AttackWeaponPicked(pc, WeaponType.Paper));
        }
        else if (10 <= rand && rand < 20)
        {
            _EventBus.Publish<AttackWeaponPicked>(new AttackWeaponPicked(pc, WeaponType.Rock));
        }
        else
        {
            _EventBus.Publish<AttackWeaponPicked>(new AttackWeaponPicked(pc, WeaponType.Scissor));
        }
    }

    protected override void PostAttackPhase(bool isWinner)
    {
    }
}

Base Greedy AI

All greedy AI extend the base greedy AI (Mid_GreedyAttacker, Mid_GreedyDefender, Mid_GreedyMixed)

public abstract class BaseMid_Greedy : BaseAI
{
    protected PlayerController opponentPc;
    protected bool canBuyUpgrade = false;
    protected WeaponType upgradableWeaponType;

    protected bool cardUsed = false;

    protected WeaponAttribute attrOfInterest;
    protected CardType cardOfInterest;

    protected virtual void SetInterest()
    {

    }

    protected override void Initialize()
    {
        opponentPc = GameController.instance.GetOpponentPlayer(pc);
        SetInterest();
    }

    protected override void BuyPhase()
    {
        float bestUpgradePrice = 10000;
        print(canBuyUpgrade);
        foreach (WeaponType type in Enum.GetValues(typeof(WeaponType)))
        {
            if (pc.CanUpgradeWeapon(type, attrOfInterest))
            {
                float upgradePrice = pc.weapons[type].GetUpgradePrice(attrOfInterest);
                if (upgradePrice < bestUpgradePrice)
                {
                    bestUpgradePrice = upgradePrice;
                    canBuyUpgrade = true;
                    upgradableWeaponType = type;
                }
            }
        }
        if (!canBuyUpgrade && pc.coins > 4)
        {
            _EventBus.Publish<CardPurchased>(new CardPurchased(pc, cardOfInterest));
        }
    }

    protected override void ActionPhase()
    {
        if (canBuyUpgrade)
        {
            _EventBus.Publish<WeaponUpgraded>(new WeaponUpgraded(pc, upgradableWeaponType, attrOfInterest));
        }
        else
        {
            if (pc.cards.Count != 0)
            {
                _EventBus.Publish<CardUsed>(new CardUsed(pc, cardOfInterest));
                cardUsed = true;
            }
        }
    }

    protected override void AttackPhase()
    {
        float highestDamage = -1000;
        float cardDamageAdd = 0;
        WeaponType attackType = WeaponType.Scissor;

        if (cardUsed)
        {
            cardDamageAdd = 0.5f;
        }

        foreach (WeaponType type in Enum.GetValues(typeof(WeaponType)))
        {
            float currDamage = pc.GetWeapon(type).baseAttack + cardDamageAdd - opponentPc.GetWeapon(pc.GetWeapon(type).GetStrongType()).baseDefense;
            if (currDamage > highestDamage)
            {
                highestDamage = currDamage;
                attackType = type;
            }
        }
        _EventBus.Publish<AttackWeaponPicked>(new AttackWeaponPicked(pc, attackType));
    }

    protected override void PostAttackPhase(bool isWinner)
    {
        canBuyUpgrade = false;
        cardUsed = false;
    }
}

Greedy Attacker (Extends BaseGreedy)

public class Mid_GreedyAttacker : BaseMid_Greedy
{
    protected override void SetInterest()
    {
        attrOfInterest = WeaponAttribute.Attack;
        cardOfInterest = CardType.SelfAttackIncreaseAdditiveCurrent1;
    }
}

Greedy Mixed (Extends BaseGreedy)

public class Mid_GreedyMixed : BaseMid_Greedy
{
    protected override void SetInterest()
    {
        return;
    }

    protected override void BuyPhase()
    {
        RandomizeInterest();
        base.BuyPhase();
    }

    private void RandomizeInterest()
    {
        int type = randObj.Next(0, 2);
        if (type == 0)
        {
            attrOfInterest = WeaponAttribute.Defense;
            cardOfInterest = CardType.SelfDefenseIncreaseAdditiveCurrent1;
        } else
        {
            attrOfInterest = WeaponAttribute.Attack;
            cardOfInterest = CardType.SelfAttackIncreaseAdditiveCurrent1;
        }
    }
}

More Changes

Super Rock Paper Scissors now uses the Pub/Sub system to allow more flexibility for logging, probing, and AI scripting.

Cards can now be bought and used in UI view.

Added a menu scene where player types can be changed and animations can be turned off.

Webbuild is now available (Check the Links section below).

Links

Github repo for Super Rock Paper Scissors: Click Here

Webbuild: Click Here

Current Project Code Structure Of Super Rock Paper Scissors

Weekly Time Spent On Project SeeSaw

Day	Hours Spent
Wed	3:00
Thu	0
Fri	0
Sat	0
Sun	4:00
Mon	4:00
Tue	0
Wed	3:00
TOTAL	14:00

Designing a Game With Balance In Mind Part 2

2020-10-07T14:30:37+00:00

This week has been a busy week, so the update will be brief

Polishing the Super Rock Paper Scissors Code

I’ve spent some time converting all internal state changes to be done through the pub/sub system, so that more internal actions can be controlled by external tools.

Card system is now semi-functional, players and AI can play and buy cards alike.

Turn-Based AI Research

Along with the Super Rock Paper Scissor’s development, I looked into how I can best write Turn-Based AI for the game.

Turn-Based RPG AI

Gives a good idea of turn-based game structure and a brief example of simple Turn-Based RPG AI.

Designing AI Algorithms For Video Games

Link

Talks about a scoring system can be used to prioritize certain actions that AI may choose. This is especially useful since I want to be able to alter how much the AI values an action over the others.

Beginner’s Guide To Game AI

Link

Explains what Game AI is and what it is not and goes over some common design patterns for creating Game AI.

It mentions that the Game AI has 3 steps: Sense, Think, and Act. Mentions that the AI is supposed to provide entertainment and challange rather than be optimal, and that the AI should make human-like decisions and feel natural.

While I will be writing AI that stress-tests the game by following extreme strategies, I also want to make sure I have AI that makes human-like decisions to monitor the game-play without needing to playtest with real players.

Explains AI decision making from several design patterns. Goes over decision trees, finite state machines, hierarchical state machines, behavior trees, utility-based systems, planning and pathfinding AI options, weight-based adaptation, markov models, and n-grams.

Citations

1) https://homes.cs.washington.edu/~zoran/jaffe2012ecg.pdf

2) https://github.com/bahaokten/Research_RPC)https://www.gamasutra.com/view/feature/129959/designing_ai_algorithms_for_.php

3) https://www.gamedev.net/tutorials/programming/artificial-intelligence/the-total-beginners-guide-to-game-ai-r4942/

Weekly Time Spent On Project SeeSaw

Day	Hours Spent
Thu	4:00
Fri	0
Sat	0
Sun	0:00
Mon	2:00
Tue	0:00
Wed	3:00
TOTAL	9:00

Designing a Game With Balance In Mind

2020-09-30T16:30:37+00:00

What it takes to make a “balancable” game

Researching the tool, techniques and design patters used for game balance revealed that before any of these methods are used, the game first has to be designed to be “balancable”.

On that note, while designing Super Rock Paper Scissors, I had to think carefully on how I can make the game’s infrastucture probable by the tools, and techniques I will be using to balance the game.

Concise and Independent Variables

Variables critical to gameplay had to be well-defined and each variable ideally had to affect the gameplay in a single independent factor (as in critical variables should not depend on each other’s value too much). I predict that keeping variables concise and indepent will make it easier to isolate certain gameplay mechanics and focus the game balance tools on each mechanic one by one. (Something similar to what is suggested in [1])

A Secondary User Interface

For balancing Super Rock Paper Scissors I wanted the game to be fully playable without a GUI, so that the game can also be played by the AI. I created an API that allows one or both players to be controlled by AI.

This means that the game can be played in couple of states:

AI vs AI, No GUI (Work In Progress, I plan on creating a logging/spectating service to dump the log game progress in to a file and analyze game balance changes)
AI vs AI, Animations On (Spectate mode, watch 2 AIs compete in a game of Super Rock Paper Scissors)
AI vs User, Animations On (Hybrid mode, animations on means the transitions are slow and animated)
AI vs User, Animations OFF (Hybrid mode, animations off means the transitions are fast)
User vs User, (This mode doesn’t make much sense in a competetive setting since both players can see each others cards and actions. The mode exists solely for testing purposes.)

AI plays the game using the PlayerAPI. The PlayerAPI has similar capabilities to pressing buttons on game’s UI.

Current Project Code Structure

How to Play Super Rock Paper Scissors? (Updated)

Round Structure

You start the game with 5 coins and 0 score.

Each player round consists of 3 phases:

Buy: In this phase you are allowed to buy as many cards as you want as long as you have the money for it.
Action: In this phase you can either use a single card or upgrade one of your weapons’ defense or attack attributes.
Attack: In this phase you pick a weapon to attack your opponent. The attack initiates one both players finish the 3 phases.

Winning Rounds and Score System

You win the round if you correctly pick the weapon that counters your opponent’s weapon. For example, if you picked scissor and your opponent picked rock, you win the round and gain +1 score. However, if you both pick the same weapon, then the player with the highest attack attribute on their weapon wins.

The first person to reach a certain score wins the game.

Coins System

Coins are earned based on the damage you deal to your opponent on a winning round. Damage is calculated as your opponent's weapon's defense attribute - your weapon's attack attribute + 1. If the damage dealt is less than 1, it is rounded up to 1.

Action Cards

There are many action cards you can buy and play.

Some examples include:

Add 0.5 attack score to one of your weapons
Add 0.5 defense score to one of your weapons
Decrease one of opponent’s weapon’s defense score by 0.5 defense next turn
Decrease one of opponent’s weapon’s defense score to 50% next turn
Stun the enemy next turn if you win the interaction
Cleanse any effects you may be under next turn
If you win the next turn, gain 50% extra coins

You gain a +0.5 attack score card whenever you reach a score value multiple of 5.

How it differs from Rock Paper Scissors

In tradinitional rock paper scissors, the biggest factor in winning the game is having good luck. A good rock paper scissors will aim to have a perfectly randomized patters while trying to guess the other person’s habits. Super Rock Paper Scissors aims to amplify the guessing stage by altering the relative power dynamics within rock, paper, scissor. For example, when a player invests coins to upgrade their rock, they will have a winning matchup against your rock. They might have a bigger incentive to play rock more often since their rock now has 2 winning matchups out of 3 possible.

Update On The Metroid-Like Platformer

Refactored old unfinished platformer game to make it compatible with balancing tools.

Links

Github repo for Super Rock Paper Scissors: Click Here

Webbuild: COMING SOON

Citations

1) https://homes.cs.washington.edu/~zoran/jaffe2012ecg.pdf

Also, RIP 30 minutes of Unity scene changes :/

Weekly Time Spent On Project SeeSaw

Day	Hours Spent
Thu	0
Fri	0
Sat	0
Sun	2:30
Mon	5:00
Tue	5:30
Wed	1:00
TOTAL	14:00