diff --git a/CONTRIBUTIONS.md b/CONTRIBUTIONS.md new file mode 100644 index 0000000..cca2da7 --- /dev/null +++ b/CONTRIBUTIONS.md @@ -0,0 +1,68 @@ +# Contributing + +If you are a new contributor, please first discuss the change you wish to make via + an issue, reddit or discord before making a change. + +## Python contributions + +Python code is formatted using [black](https://github.com/psf/black) with the settings `-l 120`. +This is not enforced, and contributions with incorrect formatting will be accepted, but formatting this way is appreciated. + +## Translations + +Adding a translation requires making a new `.po` file with entries for that languages. + +* Copy the [English .po file](https://github.com/sanderland/katrain/blob/master/katrain/i18n/locales/en/LC_MESSAGES/katrain.po) +* Change all the `msgstr` entries to your target language. + * Note that anything between `{}` should be left as-is. + * The information at the top of the file should also not be translated. + +You can send me the resulting `.po` file and I will integrate it into the program. + +For those who have some python experience as well, you can: + +* Replicate the directory structure `katrain/i18n//LC_MESSAGES/katrain.po`. +* Run `python i18.py` which will generate the `.mo` files and check for any errors. +* Find `gui.kv` and add a button near the others: + ``` + LangButton: + icon: 'img/flaticon/flag-.png' + on_press: app.language = '' + ``` +* The language should now show up in the app. +* Pull request your changes to the latest minor version branch. + +# Contributors + +## Primary author and project maintainer: + +[Sander Land](https://github.com/sanderland/) + +## Contributors + +Many thanks to these additional authors: + +* Matthew Allred ("Kameone") for design of the v1.1 UI, MacOS installation instructions, and working on promotion and YouTube videos. +* "bale-go" for development and continued work on the 'calibrated rank' AI and rank estimation algorithm. +* "Dontbtme" for detailed feedback and early testing of v1.0+. +* "nowoowoo" for a fix to the parser for SGF files with extra line breaks. +* "nimets123" for the timer sound effects and board/stone graphics. +* Jordan Seaward for the stone sound effects. +* "fohristiwhirl" for the Gibo format parsing code. + +## Translators + +Many thanks to the following contributors for translations. + +* French: "Dontbtme" with contributions from "wonderingabout" +* Korean: "isty2e" +* German: "nimets123" +* Spanish: Sergio Villegas ("serpiente") with contributions from the Spanish OGS community +* Russian: Dmitry Ivankov and Alexander Kiselev +* Simplified Chinese: Qing Mu with contributions from "Medwin" + +## Additional thanks to + +* David Wu ("lightvector") for creating KataGo and providing assistance with making the most of KataGo's amazing capabilities. +* "세븐틴" for including KaTrain in the Baduk Megapack and making explanatory YouTube videos in Korean. + diff --git a/CONTRIBUTORS b/CONTRIBUTORS deleted file mode 100644 index 43d2745..0000000 --- a/CONTRIBUTORS +++ /dev/null @@ -1,23 +0,0 @@ -Primary author and project maintainer (https://github.com/sanderland/katrain): -Sander Land - -Many thanks to these additional authors: -Matthew Allred ("Kameone") for design of the v1.1 UI, many suggestions, providing MacOS installation instructions, and working on promotion and youtube videos. -"isty2e" for the Korean translation. -"Dontbtme" for detailed feedback and early testing of v1.0, and the French translation. -"wonderingabout" for helping with the French translation. -"bale-go" for implementing and calibrating the 'calibrated rank' AI. -"nimets123" for the German translation and sound effects. -"nowoowoo" for a fix to the parser for SGF files with extra line breaks. -Sergio Villegas ("serpiente") for the Spanish translation. -Dmitry Ivankov and Alexander Kiselev for the Russian translation. -Qing Mu for the simplified Chinese translation. -"Medwin" for checking technical terms and suggesting improvements for the simplified Chinese translation. -"fohristiwhirl" for the Gibo format parsing code. - -Additional thanks to: -"Seventeen" for explanatory youtube videos in Korean. - - - - diff --git a/INSTALL.md b/INSTALL.md index 85b3500..58d943c 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,87 +1,124 @@ +# KaTrain Installation -# Installation for MacOS users +* [Quick install guide for MacOS](#MacQuick) + * [Troubleshooting and installation from sources](#MacSources) +* [Quick install guide for Windows](#WindowsQuick) + * [Troubleshooting and installation from sources](#WindowsSources) +* [Quick install guide for Linux](#LinuxQuick) + * [Troubleshooting and installation from sources](#LinuxSources) -## Installation pre-requisites -* Install [Homebrew](https://brew.sh) by running the following command in terminal: - * ``` - /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" - ``` -* Install Python using [Homebrew](https://brew.sh/) by executing `brew install python3` - * Alternatively download and install [Python 3.7.5](https://www.python.org/downloads/release/python-375/) -* Install Katago using [Homebrew](https://brew.sh/) by executing `brew install katago` - * You can also follow instructions [here](https://github.com/lightvector/KataGo) to compile KataGo yourself. -* If you encounter SSL errors on downloading model files, you may need to follow [these](https://stackoverflow.com/questions/52805115/certificate-verify-failed-unable-to-get-local-issuer-certificate) instructions to fix your certificates. +## MacOs Installation for MacOS users -## Installation and running KaTrain from PyPi -* Run `pip3 install katrain` -* Run the program by executing `katrain` in a terminal. -* If you see an error about initializing KataGo: - * Open the settings dialog by clicking on the gear icon at the bottom right of the window and change the path of the 'katago' - setting under 'engine' to `/usr/local/bin/katago`, or the path where you compiled KataGo. +### Quick install guide -# Installation for Windows users +[Open a terminal](https://support.apple.com/guide/terminal/open-or-quit-terminal-apd5265185d-f365-44cb-8b09-71a064a42125/mac) and enter the following commands: +```bash +/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" +brew install python3 +brew install katago +pip3 install katrain +``` +Now you can start KaTrain by simply typing `katrain` in a terminal. -## Downloadable .exe files +These commands install [Homebrew](https://brew.sh), which simplifies installing packages, + followed by the programming language Python, the KataGo AI, and KaTrain itself. -You can find downloadable .exe files for windows [here](https://github.com/sanderland/katrain/releases). Simply download and run, everything is included. +### Troubleshooting and Installation from sources -## Installation from sources for Windows users +Installation from sources is essentially the same as for Linux, see [here](#LinuxSources), + note that you will still need to install your own KataGo, using brew or otherwise. + +If you encounter SSL errors on downloading model files, you may need to follow [these](https://stackoverflow.com/questions/52805115/certificate-verify-failed-unable-to-get-local-issuer-certificate) instructions to fix your certificates. + +## Windows Installation for Windows users + +### Quick install guide + +You can find downloadable .exe files for windows [here](https://github.com/sanderland/katrain/releases). +Simply download and run, everything is included. + +### Installation from sources * Download the repository by clicking the green *Clone or download* on this page and *Download zip*. Extract the contents. * Make sure you have a python installation, I will assume Anaconda (Python 3.7), available [here](https://www.anaconda.com/products/individual#download-section). * Open 'Anaconda prompt' from the start menu and navigate to where you extracted the zip file using the `cd ` command. * Execute the command `pip3 install .` -* Start the app by running `katrain` in the directory where you downloaded the scripts. +* Start the app by running `katrain` in the command prompt. -# Installation from sources for Linux users +## Linux Installation for Linux users + +### Quick install guide + +If you have a working Python 3.6/3.7 available, you should be able to simply: + +* Run `pip3 install katrain` +* Run the program by executing `katrain` in a terminal. + +### Installation from sources + +This section describes how to install KaTrain from source, + in case you want to run it in a local directory or have more control over the process. +It assumes you have a working Python 3.6+ installation. -* This assumed you have a working Python 3.6/3.7 installation as a default. If your default is python 2, use pip3/python3. - Kivy currently does not have a release for Python 3.8. * Open a terminal. - * Run the command `git clone https://github.com/sanderland/katrain.git` to download the repository. - * Changing directory using `cd katrain` - * Run the command `pip3 install .` to install the package globally, or use `--user` to install locally, then run the program by typing `katrain` in the terminal. - * If you prefer not to install, run without installing using `python3 -m katrain` after installing the kivy dependencies mentioned in the next section. -* A binary for KataGo is included, but if you have compiled your own, point the 'engine/katago' setting to the relevant KataGo v1.4+ binary. +* Run the command `git clone https://github.com/sanderland/katrain.git` to download the repository and + change directory using `cd katrain` +* Run the command `pip3 install .` to install the package globally, or use `--user` to install locally. +* Run the program by typing `katrain` in the terminal. + * If you prefer not to install, run without installing using `python3 -m katrain` after installing the + dependencies from `requirements.txt`. -## Troubleshooting +A binary for KataGo is included, but if you have compiled your own, press F8 to open general settings and change the + KataGo executable path to the relevant KataGo v1.4+ binary. -Older linux machines may have trouble installing, you can try to manually install dependencies to resolve some issues. +### Troubleshooting and advanced installation from sources -The following packages may help resolve missing OS packages for Kivy or KataGo. -``` +You can try to manually install dependencies to resolve some issues relating to missing dependencies, + e.g. the binary 'wheel' is not provided, KataGo is not starting, or sounds are not working. +You can also follow these instructions if you don't want to install KaTrain, and just run it locally. + +First install the following packages, which are either required for building Kivy, + or may help resolve missing dependencies for Kivy or KataGo. +```bash sudo apt-get install python3-pip build-essential git python3 python3-dev ffmpeg libsdl2-dev libsdl2-image-dev\ libsdl2-mixer-dev libsdl2-ttf-dev libportmidi-dev libswscale-dev libavformat-dev libavcodec-dev zlib1g-dev\ - libgstreamer1.0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good\ - pkg-config libgl-dev opencl-headers ocl-icd-opencl-dev python3-pygame + libgstreamer1.0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good libpulse\ + pkg-config libgl-dev opencl-headers ocl-icd-opencl-dev ``` Then, try installing python package dependencies using: +```bash +pip3 install -r requirements.txt +pip3 install screeninfo # Skip on MacOS, not working ``` -pip3 install -U cython wheel setuptools -pip3 install kivy==2.0.0rc2 kivymd==0.104.1 +In case the sound is not working, or there is no available wheel for your OS or Python version, try: +```bash +pip3 uninstall kivy +pip3 install --no-binary kivy kivy==2.0.0rc2 ``` +You can now start KaTrain by running `python3 -m katrain` In case KataGo does not start, an alternative is to go [here](https://github.com/lightvector/KataGo) and compile KataGo yourself. -# Configuring the GPU(s) KataGo uses +## Configuring the GPU(s) KataGo uses In most cases KataGo detects your configuration correctly, automatically searching for OpenCL devices and select the highest scoring device. However, if you have multiple GPUs or want to force a specific device you will need to edit the 'analysis_config.cfg' file in the KataGo folder. To see what devices are available and which one KataGo is using. Look for the following lines in the terminal after starting KaTrain: ``` - Found 3 device(s) on platform 0 with type CPU or GPU or Accelerator - Found OpenCL Device 0: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz (Intel) (score 102) - Found OpenCL Device 1: Intel(R) UHD Graphics 630 (Intel Inc.) (score 6000102) - Found OpenCL Device 2: AMD Radeon Pro 5500M Compute Engine (AMD) (score 11000102) - Using OpenCL Device 2: AMD Radeon Pro 5500M Compute Engine (AMD) OpenCL 1.2 + Found 3 device(s) on platform 0 with type CPU or GPU or Accelerator + Found OpenCL Device 0: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz (Intel) (score 102) + Found OpenCL Device 1: Intel(R) UHD Graphics 630 (Intel Inc.) (score 6000102) + Found OpenCL Device 2: AMD Radeon Pro 5500M Compute Engine (AMD) (score 11000102) + Using OpenCL Device 2: AMD Radeon Pro 5500M Compute Engine (AMD) OpenCL 1.2 ``` The above devices were found on a 2019 MacBook Pro with both an on-motherboard graphics chip, and a separate AMD Radeon Pro video card. As you can see it scores about twice as high as the Intel UHD chip and KataGo has selected it as it's sole device. You can configure KataGo to use *both* the AMD and the Intel devices to get the best performance out of the system. -* Open the 'analysis_config.cfg' file in the KataGo folder. +* Open the 'analysis_config.cfg' file in the `katrain/KataGo` folder in your python packages, or local sources. + If you can't find it, turn on `debug_level=1` in general settings and look for the command that is used to start KataGo. * Search for `numNNServerThreadsPerModel` (~line 75), uncomment the line by deleting the # and set the value to 2. The line should read `numNNServerThreadsPerModel = 2`. * Search for `openclDeviceToUseThread` (~line 117), uncomment by deleting the # and set the values to the device ID numbers identified in the terminal. From the example above, we would want to use devices 1 and 2, for the Intel and AMD GPU's, but not device 0 (the CPU). In our case, the lines should read: @@ -89,13 +126,13 @@ As you can see it scores about twice as high as the Intel UHD chip and KataGo ha openclDeviceToUseThread0 = 1 openclDeviceToUseThread1 = 2 ``` -* Run `python3 -m katrain` and confirm that KataGo is now using both devices, by +* Run `katrain` and confirm that KataGo is now using both devices, by checking the output from the terminal, which should indicate two devices being used. For example: ``` - Found 3 device(s) on platform 0 with type CPU or GPU or Accelerator - Found OpenCL Device 0: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz (Intel) (score 102) - Found OpenCL Device 1: Intel(R) UHD Graphics 630 (Intel Inc.) (score 6000102) - Found OpenCL Device 2: AMD Radeon Pro 5500M Compute Engine (AMD) (score 11000102) - Using OpenCL Device 1: Intel(R) UHD Graphics 630 (Intel Inc.) OpenCL 1.2 - Using OpenCL Device 2: AMD Radeon Pro 5500M Compute Engine (AMD) OpenCL 1.2 + Found 3 device(s) on platform 0 with type CPU or GPU or Accelerator + Found OpenCL Device 0: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz (Intel) (score 102) + Found OpenCL Device 1: Intel(R) UHD Graphics 630 (Intel Inc.) (score 6000102) + Found OpenCL Device 2: AMD Radeon Pro 5500M Compute Engine (AMD) (score 11000102) + Using OpenCL Device 1: Intel(R) UHD Graphics 630 (Intel Inc.) OpenCL 1.2 + Using OpenCL Device 2: AMD Radeon Pro 5500M Compute Engine (AMD) OpenCL 1.2 ``` diff --git a/README.md b/README.md index dd0040f..e9bbf52 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,4 @@ - -# KaTrain v1.2 +# KaTrain [![Latest Release](https://img.shields.io/github/release/sanderland/katrain?label=download)](https://github.com/sanderland/katrain/releases) [![License:MIT](https://img.shields.io/pypi/l/katrain)](https://en.wikipedia.org/wiki/MIT_License) @@ -9,6 +8,19 @@ [![Liberapay patrons](https://img.shields.io/liberapay/patrons/sanderbaduk)](https://liberapay.com/sanderbaduk/) [![Discord](https://img.shields.io/discord/417022162348802048?logo=discord)](https://discord.com/channels/417022162348802048/629446365688365067) + +* [Introduction](#intro) +* [Previews and YouTube tutorials](#preview) +* [Installation](#install) +* [Manual](#ai) + * [Play against AI](#ai) + * [Analyzing your Games](#analysis) + * [Keyboard shortcuts](#keyboard) +* [FAQ and Troubleshooting](#faq) +* [Contributing](#support) + +## Introduction + KaTrain is a tool for analyzing and playing go with AI feedback from KataGo. The original idea was to give immediate feedback on the many large mistakes we make in terms of inefficient moves, @@ -20,7 +32,7 @@ but has since grown to include a wide range of features, including: * Play against a stronger player and use the retry option instead of handicap stones. * Automatically generate focused SGF reviews which show your biggest mistakes. -## Previews and Youtube Videos +## Previews and Youtube Videos ### Play against an AI Teacher @@ -36,16 +48,13 @@ but has since grown to include a wide range of features, including: |:-----------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:| | [![New Features Video](https://i.imgur.com/gCY6hMH.png)](https://www.youtube.com/watch?v=wFl4Bab_eGM) | [![ Teaching Game Tutorial](https://i.imgur.com/jAdcSL5.png)](https://www.youtube.com/watch?v=wFl4Bab_eGM) | - - -## Installation +## Installation * See the [releases tab](https://github.com/sanderland/katrain/releases) for pre-built installers for windows. * Alternatively use `pip3 install -U katrain` to install the latest version from PyPI on any 64-bit OS. - * Note that on MacOS you will need to set up KataGo using brew, as described [here](INSTALL.md). -* See [here](INSTALL.md#MacPrereq) for detailed instructions for running from source files on Window, Linux and MacOS, - as well as setting up KataGo to use multiple GPUs. +* See [here](INSTALL.md) for detailed instructions for Window, Linux and MacOS, + as well as troubleshooting and setting up KataGo to use multiple GPUs. -## Play against AI +## Play against AI * Select the players in the main menu, or under 'New Game'. * In a teaching game, KaTrain will analyze your moves and automatically undo those that are sufficiently bad. @@ -67,13 +76,19 @@ on the board, or not output details for them in SGFs,you can do so under 'Config This section describes the available AIs, with strength based on their current OGS rankings using the default settings. +In the 'AI settings', settings which have been tested and calibrated are at the top and have a lighter color, +changing these will show an estimate of rank. +This estimate should be reasonably accurate as long as you have not changed the other settings. + * Recommended options for serious play include: * **[9p+]** **KataGo** is full KataGo, above professional level. The analysis and feedback given is always based on this full strength KataGo AI. - * **[15k - 3d]** **Calibrated Rank Bot** was calibrated on various bots (e.g. GnuGo and Pachi at different strength settings) to play a balanced game from the opening to the endgame without making serious (DDK) blunders. Further discussion can be found on [this](https://github.com/sanderland/katrain/issues/44) thread. - * **[~5k]** **ScoreLoss** is KataGo analyzing as usual, but + * **[15k - 3d]** **Calibrated Rank Bot** was calibrated on various bots (e.g. GnuGo and Pachi at different strength settings) to play a balanced + game from the opening to the endgame without making serious (DDK) blunders. Further discussion can be found + [here](https://github.com/sanderland/katrain/issues/44) and [here](https://github.com/sanderland/katrain/issues/74). + * **[8k - 4d+]** **ScoreLoss** is KataGo analyzing as usual, but choosing from potential moves depending on the expected score loss, leading to a varied style with mostly small mistakes. - * **[~4d]** **Policy** uses the top move from the policy network (it's 'shape sense' without reading). - * **[~4k]** **Policy Weighted** picks a random move weighted by the policy, leading to a varied style with mostly small mistakes, and occasional blunders due to a lack of reading. + * **[~5d]** **Policy** uses the top move from the policy network (it's 'shape sense' without reading). + * **[12k - 2d]** **Policy Weighted** picks a random move weighted by the policy, leading to a varied style with mostly small mistakes, and occasional blunders due to a lack of reading. * **[~8k]** **Blinded Policy** picks a number of moves at random and play the best move among them, being effectively 'blind' to part of the board each turn. * Options that are more on the 'fun and experimental' side include: * Variants of **Blinded Policy**, which use the same basic strategy, but with a twist.: @@ -88,7 +103,9 @@ The Engine based AIs (KataGo, ScoreLoss, KataJigo) are affected by both the mode Further technical details and discussion on these AIs can be found on [this](https://lifein19x19.com/viewtopic.php?f=10&t=17488&sid=b11e42c005bb6f4f48c83771e6a27eff) thread at the life in 19x19 forums. -## Analysis +## Analysis + +Analysis options in KaTrain allow you to explore variations and request more in-depth analysis from the engine at any point in the game. Keyboard shortcuts are shown with **[key]**. @@ -110,27 +127,34 @@ Keyboard shortcuts are shown with **[key]**. * **[a]**: Deeper analysis: Re-evaluate the position using more visits, usually resulting in a more accurate evaluation. * **[s]**: Equalize visits: Re-evaluate all currently shown next moves with the same visits as the current top move. Useful to increase confidence in the suggestions with high uncertainty. * **[d]**: Analyze all moves: Evaluate all possible next moves. This can take a bit of time even though 'fast_visits' is used, but can be useful to see how many reasonable next moves are available. + * **[spacebar]**: Turn continuous analysis on/off. This will continuously improve analysis of the current position, similar to Lizzie's 'pondering', but only when there are no other queries going on. * **[enter]** AI move. Makes the AI move for the current player regardless of current player selection. -## Keyboard and mouse shortcuts +### Rank Estimation + +A new feature in v1.3 is the rank estimation panel. This adds an extra graph which uses a statistical model to estimate the playing strength +of both players for every 80 move segment. It can be used for determining which part of your game needs extra attention, +but keep in mind the estimation is based purely on how well moves correspond to the 'shape sense' of KataGo and can be very inaccurate at times. + +## Keyboard and mouse shortcuts In addition to shortcuts mentioned above and those shown in the main menu: * **[Shift]**: Open the main menu. * **[~]** or **[ ` ]** or **[m]**: Cycles through more minimalistic UI modes. * **[p]**: Pass -* **[spacebar]**: Pause/Resume timer +* **[b]**: Pause/Resume timer * **[arrow left]** or **[z]**: Undo move. Hold shift for 10 moves at a time, or ctrl to skip to the start. * **[arrow right]** or **[x]**: Redo move. Hold shift for 10 moves at a time, or ctrl to skip to the start. * **[arrow up/down]** Switch branch, as would be expected from the move tree. * **[scroll up]**: Undo move. Only works when hovering the cursor over the board. * **[scroll down]**: Redo move. Only works when hovering the cursor over the board. * **[click on a move]**: See detailed statistics for a previous move, along with expected variation that was best instead of this move. -* **[double-click on a move]**: Navigate directly to that point in the game. +* **[double-click on a move]**: Navigate directly to just before that point in the game. * **[Ctrl-V]**: Load SGF from clipboard and do a 'fast' analysis of the game (with a high priority normal analysis for the last move). * **[Ctrl-C]**: Save SGF to clipboard. -## FAQ +## FAQ * The program is running too slowly. How can I speed it up? * Adjust the number of visits or maximum time allowed in the settings. @@ -145,9 +169,9 @@ In addition to shortcuts mentioned above and those shown in the main menu: ## Support / Contribute [![GitHub issues](https://img.shields.io/github/issues/sanderland/katrain)](https://github.com/sanderland/katrain/issues) -[![Contributors](https://img.shields.io/static/v1?label=contributors&message=12&color=dcb424)](CONTRIBUTORS) +[![Contributors](https://img.shields.io/static/v1?label=contributors&message=16&color=dcb424)](CONTRIBUTIONS.md) [![Liberapay patrons](https://img.shields.io/liberapay/patrons/sanderbaduk)](https://liberapay.com/sanderbaduk/) -[![Github sponsors](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=dcb424&link=https://github.com/sponsors/sanderland/)](https://github.com/sponsors/sanderland) +[![Github sponsors](https://img.shields.io/static/v1?label=Sponsor&message=2&logo=GitHub&color=dcb424&link=https://github.com/sponsors/sanderland/)](https://github.com/sponsors/sanderland) * Ideas, feedback, and contributions to code or translations are all very welcome. * For suggestions and planned improvements, see [open issues](https://github.com/sanderland/katrain/issues) on github to check if the functionality is already planned. diff --git a/i18n.py b/i18n.py index a76def2..b1ca000 100644 --- a/i18n.py +++ b/i18n.py @@ -1,6 +1,7 @@ import os import re import sys +import copy from collections import defaultdict import polib @@ -27,14 +28,13 @@ for lang in locales: for entry in po[lang].translated_entries(): if "TODO" in entry.comment: todos[lang].append(entry) - else: - strings_to_langs[entry.msgid][lang] = entry.msgstr + strings_to_langs[entry.msgid][lang] = entry strings_to_keys[entry.msgid][lang] = set(re.findall("{.*?}", entry.msgstr)) if entry.msgid in lang_to_strings[lang]: print("duplicate", entry.msgid, "in", lang) errors = True lang_to_strings[lang].add(entry.msgid) - if todos[lang]: + if todos[lang] and any("todo" in a for a in sys.argv): print(f"========== {lang} has {len(todos[lang])} TODO entries ========== ") for item in todos[lang]: print(item) @@ -60,14 +60,40 @@ for lang in locales: print("Message id", msgid, "found as ", strings_to_langs[msgid], "but missing in default", DEFAULT_LANG) errors = True elif DEFAULT_LANG in strings_to_langs[msgid]: - copied_msg = strings_to_langs[msgid][DEFAULT_LANG] + copied_entry = copy.copy(strings_to_langs[msgid][DEFAULT_LANG]) print("Message id", msgid, "missing in ", lang, "-> Adding it from", DEFAULT_LANG) - entry = polib.POEntry(msgid=msgid, msgstr=copied_msg, comment="TODO") - po[lang].append(entry) + if copied_entry.comment: + copied_entry.comment = f"TODO - {copied_entry.comment}" + else: + copied_entry.comment = "TODO" + po[lang].append(copied_entry) errors = True else: print(f"MISSING IN DEFAULT AND {lang}", strings_to_langs[msgid]) errors = True + + for msgid, lang_entries in strings_to_langs.items(): + if lang in lang_entries and "TODO" in lang_entries[lang].comment: + if any(e.msgstr == lang_entries[lang].msgstr for l, e in lang_entries.items() if l != lang): + todo_comment = ( + f"TODO - {lang_entries[DEFAULT_LANG].comment}" if lang_entries[DEFAULT_LANG].comment else "TODO" + ) # update todo + if ( + lang_entries[lang].msgstr != lang_entries[DEFAULT_LANG].msgstr + or lang_entries[lang].comment.replace("\n", " ") != todo_comment + ): + print( + [ + lang_entries[lang].msgstr, + lang_entries[DEFAULT_LANG].msgstr, + lang_entries[lang].comment, + todo_comment, + ] + ) + lang_entries[lang].msgstr = lang_entries[DEFAULT_LANG].msgstr # update + lang_entries[lang].comment = todo_comment + print(f"{lang}/{msgid} todo entry updated") + po[lang].save(pofile[lang]) mofile = pofile[lang].replace(".po", ".mo") po[lang].save_as_mofile(mofile) diff --git a/katrain/__main__.py b/katrain/__main__.py index 12c202c..8e5f7fc 100644 --- a/katrain/__main__.py +++ b/katrain/__main__.py @@ -3,7 +3,8 @@ import os os.environ["KCFG_KIVY_LOG_LEVEL"] = os.environ.get("KCFG_KIVY_LOG_LEVEL", "warning") -os.environ["KIVY_AUDIO"] = "sdl2" # force working audio +if "KIVY_AUDIO" not in os.environ: + os.environ["KIVY_AUDIO"] = "sdl2" # seems to be most stable / some players hard crash # next, icon from katrain.core.utils import find_package_resource, PATHS @@ -58,6 +59,10 @@ from katrain.core.constants import ( MODE_ANALYZE, HOMEPAGE, VERSION, + STATUS_ERROR, + STATUS_INFO, + PLAYING_NORMAL, + PLAYER_HUMAN, ) from katrain.gui.popups import ConfigTeacherPopup, ConfigTimerPopup, I18NPopup from katrain.core.base_katrain import KaTrainBase @@ -65,11 +70,9 @@ from katrain.core.engine import KataGoEngine from katrain.core.game import Game, IllegalMoveException, KaTrainSGF from katrain.core.sgf_parser import Move, ParseError from katrain.gui.kivyutils import * -from katrain.gui.popups import ConfigPopup, LoadSGFPopup, NewGamePopup, AIPopup +from katrain.gui.popups import ConfigPopup, LoadSGFPopup, NewGamePopup, ConfigAIPopup from katrain.gui.style import ENGINE_BUSY_COL, ENGINE_DOWN_COL, ENGINE_READY_COL, LIGHTGREY -from katrain.gui.widgets.graph import ScoreGraph -from katrain.gui.widgets.movetree import MoveTree -from katrain.gui.widgets.filebrowser import I18NFileBrowser +from katrain.gui.widgets import * from katrain.gui.badukpan import AnalysisControls, BadukPanControls, BadukPanWidget from katrain.gui.controlspanel import ControlsPanel @@ -91,6 +94,7 @@ class KaTrainGui(Screen, KaTrainBase): self.teacher_settings_popup = None self.timer_settings_popup = None + self.idle_analysis = False self.message_queue = Queue() self._keyboard = Window.request_keyboard(None, self, "") @@ -100,21 +104,30 @@ class KaTrainGui(Screen, KaTrainBase): super().log(message, level) if level == OUTPUT_KATAGO_STDERR and "ERROR" not in self.controls.status.text: if "starting" in message.lower(): - self.controls.set_status(f"KataGo engine starting...") + self.controls.set_status(f"KataGo engine starting...", STATUS_INFO) if message.startswith("Tuning"): - self.controls.set_status(f"KataGo is tuning settings for first startup, please wait." + message) + self.controls.set_status( + f"KataGo is tuning settings for first startup, please wait." + message, STATUS_INFO + ) return if "ready" in message.lower(): - self.controls.set_status(f"KataGo engine ready.") - if (level == OUTPUT_ERROR or (level == OUTPUT_KATAGO_STDERR and "error" in message.lower())) and getattr( - self, "controls", None - ): - self.controls.set_status(f"ERROR: {message}") + self.controls.set_status(f"KataGo engine ready.", STATUS_INFO) + if ( + level == OUTPUT_ERROR + or (level == OUTPUT_KATAGO_STDERR and "error" in message.lower() and "tuning" not in message.lower()) + ) and getattr(self, "controls", None): + self.controls.set_status(f"ERROR: {message}", STATUS_ERROR) @property def play_analyze_mode(self): return self.play_mode.mode + def toggle_continuous_analysis(self): + self.idle_analysis = not self.idle_analysis + if not self.idle_analysis: + self.controls.set_status("", STATUS_INFO) + self.update_state() + def start(self): if self.engine: return @@ -158,7 +171,10 @@ class KaTrainGui(Screen, KaTrainBase): # update move tree self.controls.move_tree.current_node = self.game.current_node - def update_state( + def update_state(self, redraw_board=False): # redirect to message queue thread + self("update_state", redraw_board=redraw_board) + + def _do_update_state( self, redraw_board=False ): # is called after every message and on receiving analyses and config changes # AI and Trainer/auto-undo handlers @@ -184,7 +200,10 @@ class KaTrainGui(Screen, KaTrainBase): and not (teaching_undo and cn.auto_undo is None) ): # cn mismatch stops this if undo fired. avoid message loop here or fires repeatedly. self._do_ai_move(cn) - Clock.schedule_once(lambda _dt: self.update_gui(cn, redraw_board=redraw_board), -1) + Clock.schedule_once(self.board_gui.play_stone_sound, 0) + if len(self.engine.queries) == 0 and self.idle_analysis: + self("analyze-extra", "extra", continuous=True) + Clock.schedule_once(lambda _dt: self.update_gui(cn, redraw_board=redraw_board), -1) # trigger? def update_player(self, bw, **kwargs): super().update_player(bw, **kwargs) @@ -199,7 +218,7 @@ class KaTrainGui(Screen, KaTrainBase): def _message_loop_thread(self): while True: - game, msg, *args = self.message_queue.get() + game, msg, args, kwargs = self.message_queue.get() try: self.log(f"Message Loop Received {msg}: {args} for Game {game}", OUTPUT_EXTRA_DEBUG) if game != self.game.game_id: @@ -208,28 +227,39 @@ class KaTrainGui(Screen, KaTrainBase): ) continue fn = getattr(self, f"_do_{msg.replace('-','_')}") - fn(*args) - self.update_state() + fn(*args, **kwargs) + if msg != "update_state": + self._do_update_state() except Exception as exc: self.log(f"Exception in processing message {msg} {args}: {exc}", OUTPUT_ERROR) traceback.print_exc() - def __call__(self, message, *args): + def __call__(self, message, *args, **kwargs): if self.game: if message.endswith("popup"): # gui code needs to run in main kivy thread. fn = getattr(self, f"_do_{message.replace('-', '_')}") - Clock.schedule_once(lambda _dt: fn(*args), -1) + Clock.schedule_once(lambda _dt: fn(*args, **kwargs), -1) else: # game related actions - self.message_queue.put([self.game.game_id, message, *args]) + self.message_queue.put([self.game.game_id, message, args, kwargs]) def _do_new_game(self, move_tree=None, analyze_fast=False): + self.idle_analysis = False mode = self.play_analyze_mode if (move_tree is not None and mode == MODE_PLAY) or (move_tree is None and mode == MODE_ANALYZE): self.play_mode.switch_ui_mode() # for new game, go to play, for loaded, analyze self.board_gui.animating_pv = None self.engine.on_new_game() # clear queries self.game = Game(self, self.engine, move_tree=move_tree, analyze_fast=analyze_fast) + if move_tree: + for bw, player_info in self.players_info.items(): + player_info.player_type = PLAYER_HUMAN + player_info.player_subtype = PLAYING_NORMAL + player_info.sgf_rank = move_tree.root.get_property(bw + "R") + player_info.calculated_rank = None + player_info.name = move_tree.root.get_property("P" + bw) + self.update_player(bw) self.controls.graph.initialize_from_game(self.game.root) + self.controls.rank_graph.initialize_from_game(self.game.root) self.update_state(redraw_board=True) def _do_ai_move(self, node=None): @@ -266,10 +296,10 @@ class KaTrainGui(Screen, KaTrainBase): try: self.game.play(Move(coords, player=self.next_player_info.player)) except IllegalMoveException as e: - self.controls.set_status(f"Illegal Move: {str(e)}") + self.controls.set_status(f"Illegal Move: {str(e)}", STATUS_ERROR) - def _do_analyze_extra(self, mode): - self.game.analyze_extra(mode) + def _do_analyze_extra(self, mode, **kwargs): + self.game.analyze_extra(mode, **kwargs) def _do_new_game_popup(self): self.controls.timer.paused = True @@ -311,7 +341,7 @@ class KaTrainGui(Screen, KaTrainBase): self.controls.timer.paused = True if not self.ai_settings_popup: self.ai_settings_popup = I18NPopup( - title_key="ai settings", size=[dp(600), dp(600)], content=AIPopup(self) + title_key="ai settings", size=[dp(600), dp(650)], content=ConfigAIPopup(self) ).__self__ self.ai_settings_popup.content.popup = self.ai_settings_popup self.ai_settings_popup.open() @@ -352,15 +382,15 @@ class KaTrainGui(Screen, KaTrainBase): def _do_output_sgf(self): msg = self.game.write_sgf(self.config("general/sgf_save")) self.log(msg, OUTPUT_INFO) - self.controls.set_status(msg) + self.controls.set_status(msg, OUTPUT_INFO) def load_sgf_from_clipboard(self): clipboard = Clipboard.paste() if not clipboard: - self.controls.set_status(f"Ctrl-V pressed but clipboard is empty.") + self.controls.set_status(f"Ctrl-V pressed but clipboard is empty.", STATUS_ERROR) return try: - move_tree = KaTrainSGF.parse(clipboard) + move_tree = KaTrainSGF.parse_sgf(clipboard) except Exception as exc: self.controls.set_status( i18n._("Failed to import from clipboard").format(error=exc, contents=clipboard[:50]) @@ -431,8 +461,10 @@ class KaTrainGui(Screen, KaTrainBase): elif keycode[1] == "shift": self.nav_drawer.set_state("toggle") elif keycode[1] == "spacebar": + self.toggle_continuous_analysis() + elif keycode[1] == "b" and "ctrl" not in modifiers: self.controls.timer.paused = not self.controls.timer.paused - elif keycode[1] in ["`", "~", "m"]: + elif keycode[1] in ["`", "~", "m"] and "ctrl" not in modifiers: self.zen = (self.zen + 1) % 3 elif keycode[1] in ["left", "z"]: self("undo", 1 + ("shift" in modifiers) * 9 + ("ctrl" in modifiers) * 999) @@ -446,7 +478,7 @@ class KaTrainGui(Screen, KaTrainBase): self("output-sgf") elif keycode[1] == "c" and "ctrl" in modifiers: Clipboard.copy(self.game.root.sgf()) - self.controls.set_status(i18n._("Copied SGF to clipboard.")) + self.controls.set_status(i18n._("Copied SGF to clipboard."), STATUS_INFO) elif keycode[1] == "v" and "ctrl" in modifiers: self.load_sgf_from_clipboard() elif keycode[1] in shortcuts.keys() and "ctrl" not in modifiers: @@ -476,11 +508,10 @@ class KaTrainApp(MDApp): kv_file = find_package_resource("katrain/gui.kv") popup_kv_file = find_package_resource("katrain/popups.kv") resource_add_path(PATHS["PACKAGE"]) - Builder.load_file(kv_file) Window.bind(on_request_close=self.on_request_close) - Window.bind(on_dropfile=lambda win, file: self.gui.load_sgf_file(file)) + Window.bind(on_dropfile=lambda win, file: self.gui.load_sgf_file(file.decode("utf8"))) self.gui = KaTrainGui() Builder.load_file(popup_kv_file) @@ -493,7 +524,7 @@ class KaTrainApp(MDApp): self.gui.save_config() if self.gui.game: self.gui.update_state() - self.gui.controls.set_status("") + self.gui.controls.set_status("", STATUS_INFO) def webbrowser(self, site_key): websites = {"homepage": HOMEPAGE + "#manual", "support": HOMEPAGE + "#support"} diff --git a/katrain/config.json b/katrain/config.json index 7d0f667..9b9fedf 100644 --- a/katrain/config.json +++ b/katrain/config.json @@ -16,7 +16,7 @@ "anim_pv_time": 0.5, "debug_level": 0, "lang": "en", - "version": "1.2.2" + "version": "1.3.0" }, "timer": { "byo_length": 30, @@ -69,24 +69,28 @@ }, "ai": { "ai:default": {}, + "ai:handicap": { + "automatic": true, + "pda": 0 + }, "ai:jigo": { "target_score": 0.5 }, "ai:scoreloss": { - "strength": 0.5 + "strength": 0.2 }, "ai:policy": { "opening_moves": 22.0 }, "ai:p:weighted": { + "weaken_fac": 1.25, "pick_override": 1.0, - "lower_bound": 0.001, - "weaken_fac": 1.25 + "lower_bound": 0.001 }, "ai:p:pick": { "pick_override": 0.95, "pick_n": 5, - "pick_frac": 0.33 + "pick_frac": 0.35 }, "ai:p:local": { "pick_override": 0.95, @@ -99,13 +103,13 @@ "pick_override": 0.85, "stddev": 7.5, "pick_n": 5, - "pick_frac": 0.5, + "pick_frac": 0.4, "endgame": 0.45 }, "ai:p:influence": { "pick_override": 0.95, "pick_n": 5, - "pick_frac": 0.4, + "pick_frac": 0.3, "threshold": 3.5, "line_weight": 10, "endgame": 0.4 @@ -113,7 +117,7 @@ "ai:p:territory": { "pick_override": 0.95, "pick_n": 5, - "pick_frac": 0.4, + "pick_frac": 0.3, "threshold": 3.5, "line_weight": 2, "endgame": 0.4 diff --git a/katrain/core/ai.py b/katrain/core/ai.py index ffcbb59..eefb3da 100644 --- a/katrain/core/ai.py +++ b/katrain/core/ai.py @@ -2,27 +2,91 @@ import heapq import math import random import time -from typing import Dict, List, Tuple +from typing import Dict, List, Optional, Tuple -from katrain.core.utils import var_to_grid from katrain.core.constants import ( - OUTPUT_INFO, - OUTPUT_DEBUG, - AI_STRATEGIES_POLICY, - AI_POLICY, - AI_WEIGHTED, - AI_STRATEGIES_PICK, - AI_JIGO, - AI_SCORELOSS, AI_DEFAULT, + AI_HANDICAP, AI_INFLUENCE, + AI_JIGO, AI_LOCAL, + AI_PICK, + AI_POLICY, + AI_RANK, + AI_SCORELOSS, + AI_STRATEGIES_PICK, + AI_STRATEGIES_POLICY, + AI_STRENGTH, AI_TENUKI, AI_TERRITORY, - AI_PICK, - AI_RANK, + AI_WEIGHTED, + OUTPUT_DEBUG, + OUTPUT_ERROR, + OUTPUT_INFO, + AI_WEIGHTED_ELO, + AI_SCORELOSS_ELO, + CALIBRATED_RANK_ELO, + AI_LOCAL_ELO_GRID, + AI_TENUKI_ELO_GRID, + AI_TERRITORY_ELO_GRID, + AI_INFLUENCE_ELO_GRID, + AI_PICK_ELO_GRID, ) from katrain.core.game import Game, GameNode, Move +from katrain.core.utils import var_to_grid + + +def interp_ix(lst, x): + i = 0 + while i + 1 < len(lst) - 1 and lst[i + 1] < x: + i += 1 + t = max(0, min(1, (x - lst[i]) / (lst[i + 1] - lst[i]))) + return i, t + + +def interp1d(lst, x): + xs, ys = zip(*lst) + i, t = interp_ix(xs, x) + return (1 - t) * ys[i] + t * ys[i + 1] + + +def interp2d(gridspec, x, y): + xs, ys, matrix = gridspec + i, t = interp_ix(xs, x) + j, s = interp_ix(ys, y) + return ( + matrix[j][i] * (1 - t) * (1 - s) + + matrix[j][i + 1] * t * (1 - s) + + matrix[j + 1][i] * (1 - t) * s + + matrix[j + 1][i + 1] * t * s + ) + + +def ai_rank_estimation(strategy, settings) -> int: + if strategy in [AI_DEFAULT, AI_HANDICAP, AI_JIGO]: + return 9 + if strategy == AI_RANK: + return 1 - settings["kyu_rank"] + if strategy in [AI_WEIGHTED, AI_SCORELOSS, AI_LOCAL, AI_TENUKI, AI_TERRITORY, AI_INFLUENCE, AI_PICK]: + if strategy == AI_WEIGHTED: + elo = interp1d(AI_WEIGHTED_ELO, settings["weaken_fac"]) + if strategy == AI_SCORELOSS: + elo = interp1d(AI_SCORELOSS_ELO, settings["strength"]) + if strategy == AI_PICK: + elo = interp2d(AI_PICK_ELO_GRID, settings["pick_frac"], settings["pick_n"]) + if strategy == AI_LOCAL: + elo = interp2d(AI_LOCAL_ELO_GRID, settings["pick_frac"], settings["pick_n"]) + if strategy == AI_TENUKI: + elo = interp2d(AI_TENUKI_ELO_GRID, settings["pick_frac"], settings["pick_n"]) + if strategy == AI_TERRITORY: + elo = interp2d(AI_TERRITORY_ELO_GRID, settings["pick_frac"], settings["pick_n"]) + if strategy == AI_INFLUENCE: + elo = interp2d(AI_INFLUENCE_ELO_GRID, settings["pick_frac"], settings["pick_n"]) + + kyu = interp1d(CALIBRATED_RANK_ELO, elo) + return 1 - kyu + else: + return AI_STRENGTH[strategy] def weighted_selection_without_replacement(items: List[Tuple], pick_n: int) -> List[Tuple]: @@ -48,11 +112,12 @@ def policy_weighted_move(policy_moves, lower_bound, weaken_fac): ] if weighted_coords: top = weighted_selection_without_replacement(weighted_coords, 1)[0] - ai_thoughts = f"Playing policy-weighted random move {top[2].gtp()} ({top[0]:.1%}) from {len(weighted_coords)} moves above lower_bound of {lower_bound:.1%}." + move = top[2] + ai_thoughts = f"Playing policy-weighted random move {move.gtp()} ({top[0]:.1%}) from {len(weighted_coords)} moves above lower_bound of {lower_bound:.1%}." else: - top = policy_moves[0] + move = policy_moves[0][1] ai_thoughts = f"Playing top policy move because no non-pass move > above lower_bound of {lower_bound:.1%}." - return top[2], ai_thoughts + return move, ai_thoughts def generate_influence_territory_weights(ai_mode, ai_settings, policy_grid, size): @@ -93,8 +158,51 @@ def generate_local_tenuki_weights(ai_mode, ai_settings, policy_grid, cn, size): return weighted_coords, ai_thoughts +def request_ai_analysis(game: Game, cn: GameNode, extra_settings: Dict) -> Optional[Dict]: + error = False + analysis = None + + def set_analysis(a): + nonlocal analysis + analysis = a + + def set_error(a): + nonlocal error + game.katrain.log("Error in PDA-based analysis", a) + error = True + + engine = game.engines[cn.player] + engine.request_analysis( + cn, + callback=set_analysis, + error_callback=set_error, + priority=1_000, + ownership=False, + extra_settings=extra_settings, + ) + while not (error or analysis): + time.sleep(0.01) + engine.check_alive(exception_if_dead=True) + return analysis + + def generate_ai_move(game: Game, ai_mode: str, ai_settings: Dict) -> Tuple[Move, GameNode]: cn = game.current_node + + if ai_mode == AI_HANDICAP: + pda = ai_settings["pda"] + if ai_settings["automatic"]: + n_handicaps = len(game.root.get_list_property("AB", [])) + MOVE_VALUE = 14 # could be rules dependent + b_stones_advantage = max(n_handicaps - 1, 0) - (cn.komi - MOVE_VALUE / 2) / MOVE_VALUE + pda = min(3, max(-3, -b_stones_advantage * (3 / 8))) # max PDA at 8 stone adv, normal 9 stone game is 8.46 + handicap_analysis = request_ai_analysis( + game, cn, {"playoutDoublingAdvantage": pda, "playoutDoublingAdvantagePla": "BLACK"} + ) + if not handicap_analysis: + game.katrain.log(f"Error getting handicap-based move", OUTPUT_ERROR) + ai_mode = AI_DEFAULT + while not cn.analysis_ready: time.sleep(0.01) game.engines[cn.next_player].check_alive(exception_if_dead=True) @@ -128,12 +236,19 @@ def generate_ai_move(game: Game, ai_mode: str, ai_settings: Dict) -> Tuple[Move, board_squares = size[0] * size[1] if ai_mode == AI_RANK: # calibrated, override from 0.8 at start to ~0.4 at full board override = 0.8 * (1 - 0.5 * (board_squares - len(legal_policy_moves)) / board_squares) + overridetwo = 0.85 + max(0, 0.02 * (ai_settings["kyu_rank"] - 8)) else: override = ai_settings["pick_override"] + overridetwo = 1.0 if policy_moves[0][0] > override: aimove = top_policy_move ai_thoughts += f"Top policy move has weight > {override:.1%}, so overriding other strategies." + elif policy_moves[0][0] + policy_moves[1][0] > overridetwo: + aimove = top_policy_move + ai_thoughts += ( + f"Top two policy moves have cumulative weight > {overridetwo:.1%}, so overriding other strategies." + ) elif ai_mode == AI_WEIGHTED: aimove, ai_thoughts = policy_weighted_move( policy_moves, ai_settings["lower_bound"], ai_settings["weaken_fac"] @@ -141,9 +256,30 @@ def generate_ai_move(game: Game, ai_mode: str, ai_settings: Dict) -> Tuple[Move, elif ai_mode in AI_STRATEGIES_PICK: if ai_mode != AI_RANK: - n_moves = int(ai_settings["pick_frac"] * len(legal_policy_moves) + ai_settings["pick_n"]) + n_moves = max(1, int(ai_settings["pick_frac"] * len(legal_policy_moves) + ai_settings["pick_n"])) else: - n_moves = int(round(board_squares / 361 * 10 ** (-0.05737 * ai_settings["kyu_rank"] + 1.9482))) + orig_calib_avemodrank = 0.063015 + 0.7624 * board_squares / ( + 10 ** (-0.05737 * ai_settings["kyu_rank"] + 1.9482) + ) + norm_leg_moves = len(legal_policy_moves) / board_squares + modified_calib_avemodrank = ( + 0.3931 + + 0.6559 + * norm_leg_moves + * math.exp( + -1 + * ( + 3.002 * norm_leg_moves * norm_leg_moves + - norm_leg_moves + - 0.034889 * ai_settings["kyu_rank"] + - 0.5097 + ) + ** 2 + ) + - 0.01093 * ai_settings["kyu_rank"] + ) * orig_calib_avemodrank + n_moves = board_squares * norm_leg_moves / (1.31165 * (modified_calib_avemodrank + 1) - 0.082653) + n_moves = max(1, round(n_moves)) if ai_mode in [AI_INFLUENCE, AI_TERRITORY, AI_LOCAL, AI_TENUKI]: if cn.depth > ai_settings["endgame"] * board_squares: @@ -151,7 +287,7 @@ def generate_ai_move(game: Game, ai_mode: str, ai_settings: Dict) -> Tuple[Move, x_ai_thoughts = ( f"Generated equal weights as move number >= {ai_settings['endgame'] * size[0] * size[1]}. " ) - n_moves = int(max(n_moves, 0.5 * len(legal_policy_moves))) + n_moves = int(max(n_moves, len(legal_policy_moves) // 2)) elif ai_mode in [AI_INFLUENCE, AI_TERRITORY]: weighted_coords, x_ai_thoughts = generate_influence_territory_weights( ai_mode, ai_settings, policy_grid, size @@ -188,10 +324,13 @@ def generate_ai_move(game: Game, ai_mode: str, ai_settings: Dict) -> Tuple[Move, raise ValueError(f"Unknown Policy-based AI mode {ai_mode}") else: # Engine based move candidate_ai_moves = cn.candidate_moves + if ai_mode == AI_HANDICAP: + candidate_ai_moves = handicap_analysis["moveInfos"] + top_cand = Move.from_gtp(candidate_ai_moves[0]["move"], player=cn.next_player) - if top_cand.is_pass: # don't play suicidal to balance score - pass when it's best + if top_cand.is_pass and ai_mode not in [AI_DEFAULT, AI_HANDICAP]: # don't play suicidal to balance score aimove = top_cand - ai_thoughts += f"Top move is pass, so passing regardless of strategy." + ai_thoughts += f"Top move is pass, so passing regardless of strategy. " else: if ai_mode == AI_JIGO: sign = cn.player_sign(cn.next_player) @@ -214,11 +353,14 @@ def generate_ai_move(game: Game, ai_mode: str, ai_settings: Dict) -> Tuple[Move, aimove = topmove[2] ai_thoughts += f"ScoreLoss strategy found {len(candidate_ai_moves)} candidate moves (best {top_cand.gtp()}) and chose {aimove.gtp()} (weight {topmove[1]:.3f}, point loss {topmove[0]:.1f}) based on score weights." else: - if ai_mode != AI_DEFAULT: + if ai_mode not in [AI_DEFAULT, AI_HANDICAP]: game.katrain.log(f"Unknown AI mode {ai_mode} or policy missing, using default.", OUTPUT_INFO) ai_thoughts += f"Strategy {ai_mode} not found or unexpected fallback." aimove = top_cand - ai_thoughts += f"Default strategy found {len(candidate_ai_moves)} moves returned from the engine and chose {aimove.gtp()} as top move" + if ai_mode == AI_HANDICAP: + ai_thoughts += f"Handicap strategy found {len(candidate_ai_moves)} moves returned from the engine and chose {aimove.gtp()} as top move. PDA based score {cn.format_score(handicap_analysis['rootInfo']['scoreLead'])} and win rate {cn.format_winrate(handicap_analysis['rootInfo']['winrate'])}" + else: + ai_thoughts += f"Default strategy found {len(candidate_ai_moves)} moves returned from the engine and chose {aimove.gtp()} as top move" game.katrain.log(f"AI thoughts: {ai_thoughts}", OUTPUT_DEBUG) played_node = game.play(aimove) played_node.ai_thoughts = ai_thoughts diff --git a/katrain/core/base_katrain.py b/katrain/core/base_katrain.py index f26e4bd..627a3de 100644 --- a/katrain/core/base_katrain.py +++ b/katrain/core/base_katrain.py @@ -5,6 +5,7 @@ import sys from kivy import Config from kivy.storage.jsonstore import JsonStore +from katrain.core.ai import ai_rank_estimation from katrain.core.constants import * from katrain.core.utils import find_package_resource @@ -12,6 +13,9 @@ from katrain.core.utils import find_package_resource class Player: def __init__(self, player="B", player_type=PLAYER_HUMAN, player_subtype=PLAYING_NORMAL, periods_used=0): self.player = player + self.sgf_rank = None + self.calculated_rank = None + self.name = "" self.update(player_type, player_subtype) self.periods_used = periods_used @@ -132,6 +136,13 @@ class KaTrainBase: def update_player(self, bw, **kwargs): self.players_info[bw].update(**kwargs) + self.update_calculated_ranks() + + def update_calculated_ranks(self): + for bw, player_info in self.players_info.items(): + if player_info.player_type == PLAYER_AI: + settings = self.config(f"ai/{player_info.strategy}") + player_info.calculated_rank = ai_rank_estimation(player_info.player_subtype, settings) def reset_players(self): self.update_player("B") diff --git a/katrain/core/constants.py b/katrain/core/constants.py index 19791be..768ad43 100644 --- a/katrain/core/constants.py +++ b/katrain/core/constants.py @@ -1,6 +1,17 @@ -VERSION = "1.2.2" +VERSION = "1.3.0" HOMEPAGE = "https://github.com/sanderland/katrain" -CONFIG_MIN_VERSION = "1.2.0" +CONFIG_MIN_VERSION = "1.3.0" # keep config files from this version + +OUTPUT_ERROR = -1 +OUTPUT_KATAGO_STDERR = -0.5 +OUTPUT_INFO = 0 +OUTPUT_DEBUG = 1 +OUTPUT_EXTRA_DEBUG = 2 + +STATUS_ANALYSIS = 1.0 # same priority for analysis/info +STATUS_INFO = 1.1 +STATUS_TEACHING = 2.0 +STATUS_ERROR = 1000.0 PLAYER_HUMAN, PLAYER_AI = "player:human", "player:ai" PLAYER_TYPES = [PLAYER_HUMAN, PLAYER_AI] @@ -11,6 +22,7 @@ GAME_TYPES = [PLAYING_NORMAL, PLAYING_TEACHING] MODE_PLAY, MODE_ANALYZE = "play", "analyze" AI_DEFAULT = "ai:default" +AI_HANDICAP = "ai:handicap" AI_SCORELOSS = "ai:scoreloss" AI_WEIGHTED = "ai:p:weighted" AI_JIGO = "ai:jigo" @@ -24,13 +36,14 @@ AI_RANK = "ai:p:rank" AI_CONFIG_DEFAULT = AI_SCORELOSS -AI_STRATEGIES_ENGINE = [AI_DEFAULT, AI_SCORELOSS, AI_JIGO] +AI_STRATEGIES_ENGINE = [AI_DEFAULT, AI_HANDICAP, AI_SCORELOSS, AI_JIGO] AI_STRATEGIES_PICK = [AI_PICK, AI_LOCAL, AI_TENUKI, AI_INFLUENCE, AI_TERRITORY, AI_RANK] AI_STRATEGIES_POLICY = [AI_WEIGHTED, AI_POLICY] + AI_STRATEGIES_PICK AI_STRATEGIES = AI_STRATEGIES_ENGINE + AI_STRATEGIES_POLICY AI_STRATEGIES_RECOMMENDED_ORDER = [ AI_DEFAULT, AI_RANK, + AI_HANDICAP, AI_SCORELOSS, AI_POLICY, AI_WEIGHTED, @@ -42,24 +55,147 @@ AI_STRATEGIES_RECOMMENDED_ORDER = [ AI_JIGO, ] - -AI_STRENGTH = { - AI_DEFAULT: "9d", - AI_POLICY: "4d", - AI_JIGO: "?d", - AI_SCORELOSS: "5k", - AI_WEIGHTED: "5k", - AI_PICK: "8k", - AI_LOCAL: "5k", - AI_TENUKI: "8k", - AI_INFLUENCE: "8k", - AI_TERRITORY: "5k", - AI_RANK: "15k - 3d", +AI_STRENGTH = { # dan ranks, backup if model is missing. TODO: remove some? + AI_DEFAULT: 9, + AI_POLICY: 5, + AI_JIGO: float("nan"), + AI_SCORELOSS: -4, + AI_WEIGHTED: -4, + AI_PICK: -7, + AI_LOCAL: -4, + AI_TENUKI: -7, + AI_INFLUENCE: -7, + AI_TERRITORY: -7, + AI_RANK: float("nan"), } +AI_OPTION_VALUES = { + "kyu_rank": [(k, f"{k}[strength:kyu]") for k in range(15, 0, -1)] + + [(k, f"{1-k}[strength:dan]") for k in range(0, -3, -1)], + "strength": [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.4, 0.5, 1], + "opening_moves": range(0, 51), + "pick_override": [0, 0.5, 0.6, 0.7, 0.8, 0.85, 0.9, 0.95, 0.99, 1], + "lower_bound": [(v, f"{v:.2%}") for v in [0, 0.0001, 0.0005, 0.001, 0.005, 0.01, 0.05]], + "weaken_fac": [x / 20 for x in range(10, 3 * 20 + 1)], + "endgame": [x / 100 for x in range(10, 80, 5)], + "pick_frac": [x / 100 for x in range(0, 101, 5)], + "pick_n": range(0, 26), + "stddev": [x / 2 for x in range(21)], + "line_weight": range(0, 11), + "threshold": [2, 2.5, 3, 3.5, 4, 4.5], + "automatic": "bool", + "pda": [(x / 10, f"{'W' if x<0 else 'B'}+{abs(x/10):.1f}") for x in range(-30, 31)], +} +AI_KEY_PROPERTIES = {"kyu_rank", "strength", "weaken_fac", "pick_frac", "pick_n", "automatic"} -OUTPUT_ERROR = -1 -OUTPUT_KATAGO_STDERR = -0.5 -OUTPUT_INFO = 0 -OUTPUT_DEBUG = 1 -OUTPUT_EXTRA_DEBUG = 2 + +CALIBRATED_RANK_ELO = [ + (-21.679482223451032, 18), + (42.60243194422105, 17), + (106.88434611189314, 16), + (171.16626027956522, 15), + (235.44817444723742, 14), + (299.7300886149095, 13), + (364.0120027825817, 12), + (428.2939169502538, 11), + (492.5758311179259, 10), + (556.8577452855981, 9), + (621.1396594532702, 8), + (685.4215736209424, 7), + (749.7034877886144, 6), + (813.9854019562865, 5), + (878.2673161239586, 4), + (942.5492302916308, 3), + (1006.8311444593029, 2), + (1071.113058626975, 1), + (1135.3949727946472, 0), + (1199.6768869623193, -1), + (1263.9588011299913, -2), + (1700, -4), +] + + +AI_WEIGHTED_ELO = [ + (0.5, 1591.5718897531551), + (1.0, 1269.9896556526198), + (1.25, 1042.25179764667), + (1.5, 848.9410084463602), + (1.75, 630.1483212024823), + (2, 575.3637091858013), + (2.5, 410.9747543504796), + (3.0, 219.8667371799533), +] + +AI_SCORELOSS_ELO = [ + (0.0, 539), + (0.05, 625), + (0.1, 859), + (0.2, 1035), + (0.3, 1201), + (0.4, 1299), + (0.5, 1346), + (0.75, 1374), + (1.0, 1386), +] + + +AI_LOCAL_ELO_GRID = [ + [0.0, 0.05, 0.1, 0.2, 0.3, 0.5, 0.75, 1.0], + [0, 5, 10, 15, 25, 50], + [ + [-204.0, 791.0, 1154.0, 1372.0, 1402.0, 1473.0, 1700.0, 1700.0], + [174.0, 1094.0, 1191.0, 1384.0, 1435.0, 1522.0, 1700.0, 1700.0], + [619.0, 1155.0, 1323.0, 1390.0, 1450.0, 1558.0, 1700.0, 1700.0], + [975.0, 1289.0, 1332.0, 1401.0, 1461.0, 1575.0, 1700.0, 1700.0], + [1344.0, 1348.0, 1358.0, 1467.0, 1477.0, 1616.0, 1700.0, 1700.0], + [1425.0, 1474.0, 1489.0, 1524.0, 1571.0, 1700.0, 1700.0, 1700.0], + ], +] +AI_TENUKI_ELO_GRID = [ + [0.0, 0.05, 0.1, 0.2, 0.3, 0.5, 0.75, 1.0], + [0, 5, 10, 15, 25, 50], + [ + [47.0, 335.0, 530.0, 678.0, 830.0, 1070.0, 1376.0, 1700.0], + [99.0, 469.0, 546.0, 707.0, 855.0, 1090.0, 1413.0, 1700.0], + [327.0, 513.0, 605.0, 745.0, 875.0, 1110.0, 1424.0, 1700.0], + [429.0, 519.0, 620.0, 754.0, 900.0, 1130.0, 1435.0, 1700.0], + [492.0, 607.0, 682.0, 797.0, 1000.0, 1208.0, 1454.0, 1700.0], + [778.0, 830.0, 909.0, 949.0, 1169.0, 1461.0, 1483.0, 1700.0], + ], +] +AI_TERRITORY_ELO_GRID = [ + [0.0, 0.05, 0.1, 0.2, 0.3, 0.5, 0.75, 1.0], + [0, 5, 10, 15, 25, 50], + [ + [34.0, 383.0, 566.0, 748.0, 980.0, 1264.0, 1527.0, 1700.0], + [131.0, 450.0, 586.0, 826.0, 995.0, 1280.0, 1537.0, 1700.0], + [291.0, 517.0, 627.0, 850.0, 1010.0, 1310.0, 1547.0, 1700.0], + [454.0, 526.0, 696.0, 870.0, 1038.0, 1340.0, 1590.0, 1700.0], + [491.0, 603.0, 747.0, 890.0, 1050.0, 1390.0, 1635.0, 1700.0], + [718.0, 841.0, 1039.0, 1076.0, 1332.0, 1523.0, 1700.0, 1700.0], + ], +] +AI_INFLUENCE_ELO_GRID = [ + [0.0, 0.05, 0.1, 0.2, 0.3, 0.5, 0.75, 1.0], + [0, 5, 10, 15, 25, 50], + [ + [217.0, 439.0, 572.0, 768.0, 960.0, 1227.0, 1449.0, 1521.0], + [302.0, 551.0, 580.0, 800.0, 1028.0, 1257.0, 1470.0, 1529.0], + [388.0, 572.0, 619.0, 839.0, 1077.0, 1305.0, 1490.0, 1561.0], + [467.0, 591.0, 764.0, 878.0, 1097.0, 1390.0, 1530.0, 1591.0], + [539.0, 622.0, 815.0, 953.0, 1120.0, 1420.0, 1560.0, 1601.0], + [772.0, 912.0, 958.0, 1145.0, 1318.0, 1511.0, 1577.0, 1623.0], + ], +] +AI_PICK_ELO_GRID = [ + [0.0, 0.05, 0.1, 0.2, 0.3, 0.5, 0.75, 1.0], + [0, 5, 10, 15, 25, 50], + [ + [-533.0, -515.0, -355.0, 234.0, 650.0, 1147.0, 1546.0, 1700.0], + [-531.0, -450.0, -69.0, 347.0, 670.0, 1182.0, 1550.0, 1700.0], + [-450.0, -311.0, 140.0, 459.0, 693.0, 1252.0, 1555.0, 1700.0], + [-365.0, -82.0, 265.0, 508.0, 864.0, 1301.0, 1619.0, 1700.0], + [-113.0, 273.0, 363.0, 641.0, 983.0, 1486.0, 1700.0, 1700.0], + [514.0, 670.0, 870.0, 1128.0, 1305.0, 1550.0, 1700.0, 1700.0], + ], +] diff --git a/katrain/core/engine.py b/katrain/core/engine.py index 875a9ff..5f82ade 100644 --- a/katrain/core/engine.py +++ b/katrain/core/engine.py @@ -5,7 +5,7 @@ import subprocess import threading import time import traceback -from typing import Callable, Optional +from typing import Callable, Optional, Dict from katrain.core.constants import OUTPUT_DEBUG, OUTPUT_ERROR, OUTPUT_EXTRA_DEBUG, OUTPUT_KATAGO_STDERR from katrain.core.game_node import GameNode @@ -101,12 +101,17 @@ class KataGoEngine: self.shutdown(finish=False) self.start() - def check_alive(self, exception_if_dead=False): + def check_alive(self, os_error="", exception_if_dead=False): ok = self.katago_process and self.katago_process.poll() is None if not ok and exception_if_dead: - raise EngineDiedException( - f"Engine died (process {self.katago_process}, poll {self.katago_process and self.katago_process.poll()}) config {self.config}" - ) + if self.katago_process: + os_error += f"status {self.katago_process and self.katago_process.poll()}" + died_msg = i18n._("Engine died unexpectedly").format(error=os_error) + self.katrain.log(died_msg, OUTPUT_ERROR) + self.katago_process = None + else: + died_msg = i18n._("Engine died unexpectedly").format(error=os_error) + raise EngineDiedException(died_msg) return ok def shutdown(self, finish=False): @@ -115,8 +120,8 @@ class KataGoEngine: while self.queries and process.poll() is None: time.sleep(0.1) if process: - process.terminate() self.katago_process = None + process.terminate() if self.stderr_thread: self.stderr_thread.join() if self.analysis_thread: @@ -134,15 +139,22 @@ class KataGoEngine: self.katrain.log(line.decode(errors="ignore").strip(), OUTPUT_KATAGO_STDERR) except Exception as e: print("ERROR in processing KataGo stderr:", line, "Exception", e) - except: + elif self.katago_process: + self.check_alive(exception_if_dead=True) + except Exception as e: + self.katrain.log(f"Exception in reading stdout {e}", OUTPUT_DEBUG) return def _analysis_read_thread(self): while self.katago_process is not None: try: line = self.katago_process.stdout.readline() + if self.katago_process and not line: + self.check_alive(exception_if_dead=True) except OSError as e: - raise EngineDiedException(i18n("Engine died unexpectedly").format(error=e)) + self.check_alive(os_error=str(e), exception_if_dead=True) + return + if b"Uncaught exception" in line: self.katrain.log(f"KataGo Engine Failed: {line.decode(errors='ignore')}", OUTPUT_ERROR) return @@ -193,7 +205,7 @@ class KataGoEngine: self.katago_process.stdin.write((json.dumps(query) + "\n").encode()) self.katago_process.stdin.flush() except OSError as e: - self.katrain.log(i18n._("Engine died unexpectedly").format(error=e), OUTPUT_ERROR) + self.check_alive(os_error=str(e), exception_if_dead=True) return # do not raise, since there's nothing to catch it def request_analysis( @@ -206,9 +218,11 @@ class KataGoEngine: time_limit=True, priority: int = 0, ownership: Optional[bool] = None, - next_move=None, + next_move: Optional[GameNode] = None, + extra_settings: Optional[Dict] = None, ): - moves = [m for node in analysis_node.nodes_from_root for m in node.move_with_placements] + moves = [m for node in analysis_node.nodes_from_root for m in node.moves] + initial_stones = analysis_node.root.placements if next_move: moves.append(next_move) if ownership is None: @@ -233,9 +247,11 @@ class KataGoEngine: "komi": analysis_node.komi, "boardXSize": size_x, "boardYSize": size_y, - "includeOwnership": ownership, + "includeOwnership": ownership and not next_move, "includePolicy": not next_move, + "initialStones": [[m.player, m.gtp()] for m in initial_stones], "moves": [[m.player, m.gtp()] for m in moves], - "overrideSettings": settings, + "overrideSettings": {**settings, **(extra_settings or {})}, } self.send_query(query, callback, error_callback, next_move) + analysis_node.analysis_visits_requested = max(analysis_node.analysis_visits_requested, visits) diff --git a/katrain/core/game.py b/katrain/core/game.py index 95e2e5b..9f8b8cf 100644 --- a/katrain/core/game.py +++ b/katrain/core/game.py @@ -5,10 +5,20 @@ import threading from datetime import datetime from typing import Dict, List, Optional, Union -from katrain.core.constants import HOMEPAGE, OUTPUT_DEBUG, OUTPUT_INFO +from kivy.clock import Clock + +from katrain.core.constants import ( + HOMEPAGE, + OUTPUT_DEBUG, + OUTPUT_INFO, + STATUS_ANALYSIS, + STATUS_INFO, + STATUS_TEACHING, + PLAYER_AI, +) from katrain.core.engine import KataGoEngine from katrain.core.game_node import GameNode -from katrain.core.lang import i18n +from katrain.core.lang import i18n, rank_label from katrain.core.sgf_parser import SGF, Move from katrain.core.utils import var_to_grid @@ -46,7 +56,7 @@ class Game: self.komi = self.root.komi handicap = int(self.root.get_property("HA", 0)) if handicap and not self.root.placements: - self.place_handicap_stones(handicap) + self.root.place_handicap_stones(handicap) else: board_size = katrain.config("game/size") self.komi = katrain.config("game/komi") @@ -59,7 +69,7 @@ class Game: ) handicap = katrain.config("game/handicap") if handicap: - self.place_handicap_stones(handicap) + self.root.place_handicap_stones(handicap) if not self.root.get_property("RU"): self.root.set_property("RU", katrain.config("game/rules")) @@ -151,8 +161,9 @@ class Game: except IllegalMoveException: self._calculate_groups() raise - played_node = self.current_node.play(move) - self.current_node = played_node + with self._lock: + played_node = self.current_node.play(move) + self.current_node = played_node if analyze: played_node.analyze(self.engines[played_node.next_player]) return played_node @@ -182,35 +193,6 @@ class Game: ix = (ordered_children.index(cn) + len(ordered_children) + direction) % len(ordered_children) self.set_current_node(ordered_children[ix]) - def place_handicap_stones(self, n_handicaps): - board_size_x, board_size_y = self.board_size - near_x = 3 if board_size_x >= 13 else min(2, board_size_x - 1) - near_y = 3 if board_size_y >= 13 else min(2, board_size_y - 1) - far_x = board_size_x - 1 - near_x - far_y = board_size_y - 1 - near_y - middle_x = board_size_x // 2 # what for even sizes? - middle_y = board_size_y // 2 - if n_handicaps > 9 and board_size_x == board_size_y: - stones_per_row = math.ceil(math.sqrt(n_handicaps)) - spacing = (far_x - near_x) / (stones_per_row - 1) - if spacing < near_x: - far_x += 1 - near_x -= 1 - spacing = (far_x - near_x) / (stones_per_row - 1) - coords = list({math.floor(0.5 + near_x + i * spacing) for i in range(stones_per_row)}) - stones = sorted( - [(x, y) for x in coords for y in coords], - key=lambda xy: -((xy[0] - (board_size_x - 1) / 2) ** 2 + (xy[1] - (board_size_y - 1) / 2) ** 2), - ) - else: # max 9 - stones = [(far_x, far_y), (near_x, near_y), (far_x, near_y), (near_x, far_y)] - if n_handicaps % 2 == 1: - stones.append((middle_x, middle_y)) - stones += [(near_x, middle_y), (far_x, middle_y), (middle_x, near_y), (middle_x, far_y)] - self.root.set_property( - "AB", list({Move(stone).sgf(board_size=(board_size_x, board_size_y)) for stone in stones[:n_handicaps]}) - ) - @property def board_size(self): return self.root.board_size @@ -297,12 +279,14 @@ class Game: def player_name(player_info): return f"{i18n._(player_info.player_type)} ({i18n._(player_info.player_subtype)})" - player_names = { - bw: re.sub( - r"['<>:\"/\\|?*]", "", self.root.get_property("P" + bw) or player_name(self.katrain.players_info[bw]) - ) - for bw in "BW" - } + if "KaTrain" in self.root.get_property("AP", ""): + for bw in "BW": + self.root.set_property("P" + bw, player_name(self.katrain.players_info[bw])) + player_info = self.katrain.players_info[bw] + if player_info.player_type == PLAYER_AI: + self.root.set_property(bw + "R", rank_label(player_info.calculated_rank)) + + player_names = {bw: re.sub(r"['<>:\"/\\|?*]", "", self.root.get_property("P" + bw, bw)) for bw in "BW"} game_name = f"katrain_{player_names['B']} vs {player_names['W']} {self.game_id}" file_name = os.path.abspath(os.path.join(path, f"{game_name}.sgf")) os.makedirs(os.path.dirname(file_name), exist_ok=True) @@ -317,16 +301,33 @@ class Game: f.write(sgf) return i18n._("sgf written").format(file_name=file_name) - def analyze_extra(self, mode): + def analyze_extra(self, mode, **kwargs): stones = {s.coords for s in self.stones} cn = self.current_node engine = self.engines[cn.next_player] + Clock.schedule_once(self.katrain.analysis_controls.hints.activate, 0) + if mode == "extra": - visits = cn.analysis_visits_requested + engine.config["max_visits"] - self.katrain.controls.set_status(i18n._("extra analysis").format(visits=visits)) + if kwargs.get("continuous", False): + visits = max(engine.config["max_visits"], math.ceil(cn.analysis_visits_requested * 1.25)) + else: + visits = cn.analysis_visits_requested + engine.config["max_visits"] + self.katrain.controls.set_status(i18n._("extra analysis").format(visits=visits), STATUS_ANALYSIS) cn.analyze(engine, visits=visits, priority=-1_000, time_limit=False) return + if mode == "game": + nodes = self.root.nodes_in_tree + if "visits" in kwargs: + visits = kwargs["visits"] + else: + min_visits = min(node.analysis_visits_requested for node in nodes) + visits = min_visits + engine.config["max_visits"] + for node in nodes: + node.analyze(engine, visits=visits, priority=-1_000_000, time_limit=False) + self.katrain.controls.set_status(i18n._("game re-analysis").format(visits=visits), STATUS_ANALYSIS) + return + elif mode == "sweep": board_size_x, board_size_y = self.board_size if cn.analysis_ready: @@ -352,17 +353,19 @@ class Game: if (x, y) not in stones ] visits = engine.config["fast_visits"] - self.katrain.controls.set_status(i18n._("sweep analysis").format(visits=visits)) + self.katrain.controls.set_status(i18n._("sweep analysis").format(visits=visits), STATUS_ANALYSIS) priority = -1_000_000_000 - else: # mode=='equalize': + elif mode == "equalize": if not cn.analysis_ready: - self.katrain.controls.set_status(i18n._("wait-before-equalize"), self.current_node) + self.katrain.controls.set_status(i18n._("wait-before-equalize"), STATUS_INFO, self.current_node) return analyze_moves = [Move.from_gtp(gtp, player=cn.next_player) for gtp, _ in cn.analysis["moves"].items()] visits = max(d["visits"] for d in cn.analysis["moves"].values()) - self.katrain.controls.set_status(i18n._("equalizing analysis").format(visits=visits)) + self.katrain.controls.set_status(i18n._("equalizing analysis").format(visits=visits), STATUS_ANALYSIS) priority = -1_000 + else: + raise ValueError("Invalid analysis mode") for move in analyze_moves: cn.analyze( engine, priority, visits=visits, refine_move=move, time_limit=False @@ -391,6 +394,6 @@ class Game: if undo: self.undo(1) self.katrain.controls.set_status( - i18n._("teaching undo message").format(move=move.gtp(), points_lost=points_lost) + i18n._("teaching undo message").format(move=move.gtp(), points_lost=points_lost), STATUS_TEACHING ) self.katrain.update_state() diff --git a/katrain/core/game_node.py b/katrain/core/game_node.py index f3eae5b..a4f8676 100644 --- a/katrain/core/game_node.py +++ b/katrain/core/game_node.py @@ -69,8 +69,6 @@ class GameNode(SGFNode): # various analysis functions def analyze(self, engine, priority=0, visits=None, time_limit=True, refine_move=None, analyze_fast=False): - if visits and not refine_move: - self.analysis_visits_requested = max(visits, engine.config["max_visits"]) engine.request_analysis( self, lambda result: self.set_analysis(result, refine_move), @@ -136,6 +134,15 @@ class GameNode(SGFNode): if win_rate is not None: return f"{'B' if win_rate > 0.5 else 'W'} {max(win_rate,1-win_rate):.1%}" + def move_policy_stats(self) -> Tuple[Optional[int], float, List]: + single_move = self.move + if single_move and self.parent: + policy_ranking = self.parent.policy_ranking + for ix, (p, m) in enumerate(policy_ranking): + if m == single_move: + return ix + 1, p, policy_ranking + return None, 0.0, [] + def make_pv(self, player, pv, interactive): pvtext = f"{player}{' '.join(pv)}" if interactive: @@ -160,44 +167,24 @@ class GameNode(SGFNode): points_lost = self.points_lost if sgf and points_lost > 0.5: text += i18n._("Info:point loss").format(points_lost=points_lost) + "\n" - text += ( - i18n._("Info:top move").format( - top_move=previous_top_move["move"], - score=self.format_score(previous_top_move["scoreLead"]), - ) - + "\n" - ) + top_move = previous_top_move["move"] + score = self.format_score(previous_top_move["scoreLead"]) + text += i18n._("Info:top move").format(top_move=top_move, score=score,) + "\n" else: text += i18n._("Info:best move") + "\n" if previous_top_move.get("pv") and (sgf or details): - text += ( - i18n._("Info:PV").format( - pv=self.make_pv(single_move.player, previous_top_move["pv"], interactive) - ) - + "\n" - ) + pv = self.make_pv(single_move.player, previous_top_move["pv"], interactive) + text += i18n._("Info:PV").format(pv=pv) + "\n" if sgf or details or teach: - policy_ranking = self.parent.policy_ranking - currmove_policy_with_ix = [ - (ix + 1, p) - for (p, m), ix in zip(policy_ranking, range(len(policy_ranking))) - if m == single_move - ] - if currmove_policy_with_ix: - text += ( - i18n._("Info:policy rank").format( - rank=currmove_policy_with_ix[0][0], probability=currmove_policy_with_ix[0][1] - ) - + "\n" - ) - if not currmove_policy_with_ix or currmove_policy_with_ix[0] != 1 and (sgf or details): - text += ( - i18n._("Info:policy best").format( - move=policy_ranking[0][1].gtp(), probability=policy_ranking[0][0] - ) - + "\n" - ) + currmove_pol_rank, currmove_pol_prob, policy_ranking = self.move_policy_stats() + if currmove_pol_rank is not None: + policy_rank_msg = i18n._("Info:policy rank") + text += policy_rank_msg.format(rank=currmove_pol_rank, probability=currmove_pol_prob) + "\n" + if currmove_pol_rank is None or currmove_pol_rank != 1 and (sgf or details): + policy_best_msg = i18n._("Info:policy best") + pol_move, pol_prob = policy_ranking[0][1].gtp(), policy_ranking[0][0] + text += policy_best_msg.format(move=pol_move, probability=pol_prob) + "\n" if self.auto_undo and sgf: text += i18n._("Info:teaching undo") + "\n" top_pv = self.analysis_ready and self.candidate_moves[0].get("pv") @@ -243,7 +230,13 @@ class GameNode(SGFNode): polmoves = self.policy_ranking top_polmove = polmoves[0][1] if polmoves else Move(None) # if no info at all, pass return [ - {**self.analysis["root"], "pointsLost": 0, "order": 0, "move": top_polmove.gtp()} + { + **self.analysis["root"], + "pointsLost": 0, + "order": 0, + "move": top_polmove.gtp(), + "pv": [top_polmove.gtp()], + } ] # single visit -> go by policy/root root_score = self.analysis["root"]["scoreLead"] diff --git a/katrain/core/lang.py b/katrain/core/lang.py index 0e286ce..3b045ef 100644 --- a/katrain/core/lang.py +++ b/katrain/core/lang.py @@ -77,3 +77,13 @@ class Lang(Observable): DEFAULT_LANGUAGE = "en" i18n = Lang(DEFAULT_LANGUAGE) + + +def rank_label(rank): + if rank is None: + return "??k" + + if rank >= 0.5: + return f"{rank:.0f}{i18n._('strength:dan')}" + else: + return f"{1-rank:.0f}{i18n._('strength:kyu')}" diff --git a/katrain/core/sgf_parser.py b/katrain/core/sgf_parser.py index 8574139..a39fe84 100644 --- a/katrain/core/sgf_parser.py +++ b/katrain/core/sgf_parser.py @@ -1,4 +1,5 @@ import copy +import math import re from collections import defaultdict from typing import Any, Dict, List, Optional, Tuple @@ -285,6 +286,37 @@ class SGFNode: else: return "W" + def place_handicap_stones(self, n_handicaps, tygem=False): + board_size_x, board_size_y = self.board_size + near_x = 3 if board_size_x >= 13 else min(2, board_size_x - 1) + near_y = 3 if board_size_y >= 13 else min(2, board_size_y - 1) + far_x = board_size_x - 1 - near_x + far_y = board_size_y - 1 - near_y + middle_x = board_size_x // 2 # what for even sizes? + middle_y = board_size_y // 2 + if n_handicaps > 9 and board_size_x == board_size_y: + stones_per_row = math.ceil(math.sqrt(n_handicaps)) + spacing = (far_x - near_x) / (stones_per_row - 1) + if spacing < near_x: + far_x += 1 + near_x -= 1 + spacing = (far_x - near_x) / (stones_per_row - 1) + coords = list({math.floor(0.5 + near_x + i * spacing) for i in range(stones_per_row)}) + stones = sorted( + [(x, y) for x in coords for y in coords], + key=lambda xy: -((xy[0] - (board_size_x - 1) / 2) ** 2 + (xy[1] - (board_size_y - 1) / 2) ** 2), + ) + else: # max 9 + stones = [(far_x, far_y), (near_x, near_y), (far_x, near_y), (near_x, far_y)] + if n_handicaps % 2 == 1: + stones.append((middle_x, middle_y)) + stones += [(near_x, middle_y), (far_x, middle_y), (middle_x, near_y), (middle_x, far_y)] + if tygem: + stones[2], stones[3] = stones[3], stones[2] + self.set_property( + "AB", list({Move(stone).sgf(board_size=(board_size_x, board_size_y)) for stone in stones[:n_handicaps]}) + ) + class SGF: @@ -293,23 +325,31 @@ class SGF: SGFPROP_PAT = re.compile(r"\s*(?:\(|\)|;|(\w+)((\s*\[([^\]\\]|\\.)*\])+))", flags=re.DOTALL) @classmethod - def parse(cls, input_str) -> SGFNode: + def parse_sgf(cls, input_str) -> SGFNode: """Parse a string as SGF.""" return cls(input_str).root @classmethod def parse_file(cls, filename, encoding=None) -> SGFNode: + is_gib = filename.lower().endswith(".gib") + """Parse a file as SGF, encoding will be detected if not given.""" with open(filename, "rb") as f: bin_contents = f.read() if not encoding: - match = re.search(rb"CA\[(.*?)\]", bin_contents) - if match: - encoding = match[1].decode("ascii", errors="ignore") + if not is_gib: + match = re.search(rb"CA\[(.*?)\]", bin_contents) + if match: + encoding = match[1].decode("ascii", errors="ignore") + else: + encoding = "ISO-8859-1" # default else: - encoding = "ISO-8859-1" # default - decoded = bin_contents.decode(encoding=encoding, errors="ignore") - return cls.parse(decoded) + encoding = "utf8" # ? + decoded = bin_contents.decode(encoding=encoding, errors="ignore") + if is_gib: + return cls.parse_gib(decoded) + else: # sgf + return cls.parse_sgf(decoded) def __init__(self, contents): self.contents = contents @@ -341,3 +381,126 @@ class SGF: if self.ix < len(self.contents): raise ParseError(f"Parse Error: unexpected character at {self.contents[self.ix:self.ix+25]}") raise ParseError("Parse Error: expected ')' at end of input.") + + # GIB parser adapted from https://github.com/fohristiwhirl/gofish/ + @classmethod + def parse_gib(cls, gib): + def parse_player_name(raw): + name = raw + rank = "" + foo = raw.split("(") + if len(foo) == 2: + if foo[1][-1] == ")": + name = foo[0].strip() + rank = foo[1][0:-1] + return name, rank + + def gib_make_result(grlt, zipsu): + easycases = {3: "B+R", 4: "W+R", 7: "B+T", 8: "W+T"} + + if grlt in easycases: + return easycases[grlt] + + if grlt in [0, 1]: + return "{}+{}".format("B" if grlt == 0 else "W", zipsu / 10) + + return "" + + def gib_get_result(line, grlt_regex, zipsu_regex): + try: + grlt = int(re.search(grlt_regex, line).group(1)) + zipsu = int(re.search(zipsu_regex, line).group(1)) + except: + return "" + return gib_make_result(grlt, zipsu) + + root = cls._NODE_CLASS() + node = root + + lines = gib.split("\n") + for line in lines: + line = line.strip() + if line.startswith("\\[GAMEBLACKNAME=") and line.endswith("\\]"): + s = line[16:-2] + name, rank = parse_player_name(s) + if name: + root.set_property("PB", name) + if rank: + root.set_property("BR", rank) + + if line.startswith("\\[GAMEWHITENAME=") and line.endswith("\\]"): + s = line[16:-2] + name, rank = parse_player_name(s) + if name: + root.set_property("PW", name) + if rank: + root.set_property("WR", rank) + + if line.startswith("\\[GAMEINFOMAIN="): + result = gib_get_result(line, r"GRLT:(\d+),", r"ZIPSU:(\d+),") + if result: + root.set_property("RE", result) + try: + komi = int(re.search(r"GONGJE:(\d+),", line).group(1)) / 10 + if komi: + root.set_property("KM", komi) + except: + pass + + if line.startswith("\\[GAMETAG="): + if "DT" not in root.properties: + try: + match = re.search(r"C(\d\d\d\d):(\d\d):(\d\d)", line) + date = "{}-{}-{}".format(match.group(1), match.group(2), match.group(3)) + root.set_property("DT", date) + except: + pass + + if "RE" not in root.properties: + result = gib_get_result(line, r",W(\d+),", r",Z(\d+),") + if result: + root.set_property("RE", result) + + if "KM" not in root.properties: + try: + komi = int(re.search(r",G(\d+),", line).group(1)) / 10 + if komi: + root.set_property("KM", komi) + except: + pass + + if line[0:3] == "INI": + if node is not root: + raise ParseError("Node is not root") + setup = line.split() + try: + handicap = int(setup[3]) + except IndexError: + continue + + if handicap < 0 or handicap > 9: + raise ParseError(f"Handicap {handicap} out of range") + + if handicap >= 2: + root.set_property("HA", handicap) + root.place_handicap_stones(handicap, tygem=True) + + if line[0:3] == "STO": + move = line.split() + key = "B" if move[3] == "1" else "W" + try: + x = int(move[4]) + y = 18 - int(move[5]) + if not (0 <= x < 19 and 0 <= y < 19): + raise ParseError(f"Coordinates for move ({x},{y}) out of range on line {line}") + value = Move(coords=(x, y)).sgf(board_size=(19, 19)) + except IndexError: + continue + + node = cls._NODE_CLASS(parent=node) + node.set_property(key, value) + + if len(root.children) == 0: # We'll assume we failed in this case + raise ParseError("No valid nodes found") + + return root diff --git a/katrain/gui.kv b/katrain/gui.kv index 3990b7b..682d1bc 100644 --- a/katrain/gui.kv +++ b/katrain/gui.kv @@ -4,8 +4,12 @@ #:import PLAYER_TYPES katrain.core.constants.PLAYER_TYPES #:import PLAYER_HUMAN katrain.core.constants.PLAYER_HUMAN #:import PLAYER_AI katrain.core.constants.PLAYER_AI +#:import PLAYING_NORMAL katrain.core.constants.PLAYING_NORMAL +#:import PLAYING_TEACHING katrain.core.constants.PLAYING_TEACHING #:import MODE_PLAY katrain.core.constants.MODE_PLAY #:import MODE_ANALYZE katrain.core.constants.MODE_ANALYZE +#:import STATUS_INFO katrain.core.constants.STATUS_INFO + #:import WHITE katrain.gui.style.WHITE #:import BLACK katrain.gui.style.BLACK @@ -30,6 +34,8 @@ #:set CP_PADDING dp(6) #:set NOTES_FONT_SIZE dp(18) +#:set EPSILON 1e-9 + # for sizing help #