status, sgf

TODO

@@ -16,34 +16,29 @@
 [x] ogs bot connector
 [x] clarify score change vs score
 [x] List edit settings/object edit settings? -> Mostly superseded by popups.
+[x] box to label ? split in status and comment?
+[x] Currently, when saving an SGF after analyzing a game, Katrain stores data for every move by default. Like for dots, it would be great if it was possible to only store data for moves that we were interested in (e.g. only from this or that player, and/or only the most inefficient moves/most costly mistakes, etc.)
+[x] Typing something in the comments window freezes Katrain -> probably keyboard shortcuts / solved by label
+[x] In the 0.9 version I could hide dots from one player, which was handy to focus on my own mistakes when reviewing games, not my opponent's. For what I saw, we can't do that in the 1.0 version anymore, which is a shame IMO.
+[x]  Since dots help to discriminate between mistakes, it would be nice to be able to show only those that are big enough to worry about: for example showing only non-green dots, or showing only orange and red dots, or even only red dots, since strong and weak players will have different needs. In my case, when I was reviewing games (my own or classical Japanese ones) with the 0.9 version of Katrain, I would typically focus on moves that would be rated less than 80% efficient (ranging from orange to red, if I remember correctly). A slider going from 0 to 100% to show moves according to their efficiency would be handy, I think. Just an idea ^_^
+[x] engine status
+[x?] sgf review improvements  --  Likewise, in the 0.9 version, better alternatives to the played move were shown with squares, which was also pretty useful when using the sgf outside of Katrain. I mean, having the top move mentioned is all and good, but when you see multiple squares shown on the board as better alternatives to the move played in the game, it makes obvious how far from perfect that move actually was :D
+
 
 [/] README
-[] engine status
-[] sgf review improvements  --  Likewise, in the 0.9 version, better alternatives to the played move were shown with squares, which was also pretty useful when using the sgf outside of Katrain. I mean, having the top move mentioned is all and good, but when you see multiple squares shown on the board as better alternatives to the move played in the game, it makes obvious how far from perfect that move actually was :D
-[] selective save dots / config
 [] Release notes
-[] box to label ? split in status and comment?
-[] pv with overlap?
 
 Low priority
+[] pv with overlap?
 [] more teaching / groups in danger? hard
 [] When creating a new game, the 9 buttons on the right side aren't all that useful. Maybe the 9, 13 and 19 ones make sense since these three board sizes are the traditionally used ones, but why 2, 4 and 9 stones buttons? Why 0.5, 6.5 and -40pts komi buttons?
 [] Score instead of game end
 
 Wont do for now
 [] dual engine support -- easily possible but has weird effects on win rate etc
-[] Currently, when saving an SGF after analyzing a game, Katrain stores data for every move by default. Like for dots, it would be great if it was possible to only store data for moves that we were interested in (e.g. only from this or that player, and/or only the most inefficient moves/most costly mistakes, etc.)
-[] Typing something in the comments window freezes Katrain -> probably keyboard shortcuts / solved by label
-[] Plus, you wouldn't need "B" and "W" beside these numbers if the top part of the window was black with the graph being white on top of it (and the bottom White with the graph being black as they are currently).
+[/] The UI would be directly understood if there were bubbles with a short explanation popping up when hovering above buttons and labels, e.g "Performing additional analysis to 1502 visits" which appears currently in the comments window after pushing the Extra button would also make a perfect explanatory bubble when hovering over said button too.
+--- partially solved by config with help for some
 
 bug/suggestion reports
-- The territory span B+something W+something was confusing: it looked like the last move was both of them and it took me a while to figure out what it was about. It would be maybe clearer if these numbers appeared as dynamic graduations on the left outside of the winrate window, like graduations on a ruler.
---- Like, say if Black was 13pts ahead at some point in the game, the ruler would be:  15 / 10 / 5 / 0 / ... -15
-
-
--- The UI would be directly understood if there were bubbles with a short explanation popping up when hovering above buttons and labels, e.g "Performing additional analysis to 1502 visits" which appears currently in the comments window after pushing the Extra button would also make a perfect explanatory bubble when hovering over said button too.
 -- Plus, explanatory bubbles would help to clarify labels like "Debug" (are there bugs in the matrix? :p), "enable_ownership" (which didn't seem to prevent or allow visualizing territory ownership on the board, as that was already covered by the "owner" checkbox anyway... so I guess I missed something), etc.
--- That would also prevent the need to go through the official documentation for every little thing, making Katrain more user friendly, IMO.
-- In the 0.9 version I could hide dots from one player, which was handy to focus on my own mistakes when reviewing games, not my opponent's. For what I saw, we can't do that in the 1.0 version anymore, which is a shame IMO.
---  Since dots help to discriminate between mistakes, it would be nice to be able to show only those that are big enough to worry about: for example showing only non-green dots, or showing only orange and red dots, or even only red dots, since strong and weak players will have different needs. In my case, when I was reviewing games (my own or classical Japanese ones) with the 0.9 version of Katrain, I would typically focus on moves that would be rated less than 80% efficient (ranging from orange to red, if I remember correctly). A slider going from 0 to 100% to show moves according to their efficiency would be handy, I think. Just an idea ^_^
 

ai.py

@@ -55,11 +55,26 @@ def ai_move(game: Game, ai_mode: str, ai_settings: Dict) -> Tuple[Move, GameNode
             aimove = top_policy_move
             ai_thoughts += f"Top policy move has weight > {ai_settings['pick_override']:.1%}, so overriding other strategies."
         elif top_5_pass or "weighted" in ai_mode:
-            weighted_coords = [(policy_grid[y][x], policy_grid[y][x], x, y) for x in range(size[0]) for y in range(size[1]) if policy_grid[y][x] > 0]
-            best = weighted_selection_without_replacement(weighted_coords, 1)[0]
-            aimove = Move(best[2:], player=cn.next_player)  # just take a random move by policy w/o noise
-            ai_thoughts += f"Playing policy-weighted random move {aimove.gtp()} ({best[0]:.1%})" + (
-                " because one of them is pass." if top_5_pass else " because strategy is weighted."
+            if top_5_pass:
+                lower_bound = 0.05
+                weaken_fac = 1
+            else:
+                lower_bound = max(0, ai_settings["lower_bound"])
+                weaken_fac = max(0.01, ai_settings["weaken_fac"])
+            weighted_coords = [(policy_grid[y][x], policy_grid[y][x] ** (1 / weaken_fac), x, y) for x in range(size[0]) for y in range(size[1]) if policy_grid[y][x] > lower_bound]
+            if top_5_pass or pass_policy > lower_bound:
+                weighted_coords.append([pass_policy, pass_policy ** (1 / weaken_fac), None, None])
+            top = weighted_selection_without_replacement(weighted_coords, 1)
+            if top and top[0][2]:
+                best = top[0]
+                policy_value = best[0]
+                coords = best[2:]
+            else:
+                policy_value = pass_policy
+                coords = None
+            aimove = Move(coords, player=cn.next_player)  # just take a random move by policy w/o noise
+            ai_thoughts += f"Playing policy-weighted random move {aimove.gtp()} ({policy_value:.1%})" + (
+                " because one of them is pass." if top_5_pass else f" because strategy is weighted (lower bound={lower_bound:.2%}, num moves > lb={len(weighted_coords)})."
             )
         elif "noise" in ai_mode:
             noise_str = ai_settings["noise_strength"]

config.json

@@ -5,12 +5,21 @@
         "config": "KataGo/analysis_config.cfg",
         "threads": 8,
         "max_visits": 500,
+        "fast_visits": 50,
         "max_time": 3.0,
         "enable_ownership": true
     },
-    "files": {
+    "sgf": {
         "sgf_load": "~/Downloads",
-        "sgf_save": "./sgfout"
+        "sgf_save": "./sgfout",
+        "save_feedback": [
+            true,
+            true,
+            true,
+            true,
+            true,
+            true
+        ]
     },
     "game": {
         "init_size": 19,
@@ -37,7 +46,7 @@
             0
         ],
         "eval_off_show_last": 3,
-        "eval_show_ai": false
+        "eval_show_ai": true
     },
     "ai": {
         "Default": {
@@ -62,9 +71,11 @@
             "_help_left": ""
         },
         "P:Weighted": {
-            "_help_right": "Strength is mainly affected by `model` in engine settings.",
-            "_help_left": "`pick_override` determines when top move is chosen without randomness, and is effectively disabled by default (1.0).",
-            "pick_override": 1.0
+            "_help_right": "`lower_bound` determines the lower bound policy value that is allowed. `weaken_fac` influences how much more likely weaker moves are picked.",
+            "_help_left": "pick_override` determines when top move is chosen without randomness, and is effectively disabled by default (1.0).",
+            "pick_override": 1.0,
+            "lower_bound": 0.001,
+            "weaken_fac": 1
         },
         "P:Noise": {
             "pick_override": 0.95,
@@ -112,14 +123,49 @@
             "line_weight": 5,
             "_help_right": "Samples `pick_n + pick_frac * <number of legal moves>` and plays the best one, biased to below the `threshold` line.",
             "_help_left": "Increase `line_weight` to penalize moves closer to the center more."
+        },
+        "<Pause>": {
+                        "_help_right": "",
+            "_help_left": ""
+
         }
     },
     "board_ui": {
+        "engine_down_col": [
+            0.8,
+            0,
+            0,
+            1
+        ],
+        "engine_busy_col": [
+            0.9,
+            0.4,
+            0.1,
+            1
+        ],
+        "engine_little_busy_col": [
+            0.85,
+            0.89,
+            0.3,
+            1
+        ],
+        "engine_almost_done_col": [
+            0.67,
+            0.9,
+            0.18,
+            1
+        ],
+        "engine_ready_col": [
+            0.117,
+            0.588,
+            0,
+            1
+        ],
         "starpoint_size": 0.1,
         "stone_size": 0.475,
         "eval_dot_max_size": 0.5,
-      "eval_dot_min_size": 0.25,
-      "visit_frac_small": 0.1,
+        "eval_dot_min_size": 0.25,
+        "visit_frac_small": 0.1,
         "stones": {
             "B": [
                 0.05,
@@ -178,13 +224,13 @@
                 0.67,
                 0.9,
                 0.18,
-                0.0
+                1.0
             ],
             [
                 0.117,
                 0.588,
                 0,
-                0.0
+                1.0
             ]
         ],
         "line_color": [

engine.py

@@ -52,7 +52,7 @@ class KataGoEngine:
         self.queries = {}
 
     def shutdown(self, finish=False):
-        process = getattr(self, "katago_process", None)
+        process = self.katago_process
         if finish and process:
             while self.queries and process.poll() is None:
                 time.sleep(0.1)
@@ -120,6 +120,7 @@ class KataGoEngine:
         callback: Callable,
         error_callback: Optional[Callable] = None,
         visits: int = None,
+        analyze_fast: bool = False,
         time_limit=True,
         priority: int = 0,
         ownership: Optional[bool] = None,
@@ -130,12 +131,17 @@ class KataGoEngine:
             moves.append(next_move)
         if ownership is None:
             ownership = self.config["enable_ownership"] and not next_move
+        if visits is None:
+            visits = self.config["max_visits"]
+            if analyze_fast and self.config.get("fast_visits"):
+                visits = self.config["fast_visits"]
+
         size_x, size_y = analysis_node.board_size
         query = {
             "rules": self.get_rules(analysis_node),
             "priority": self.base_priority + priority,
             "analyzeTurns": [len(moves)],
-            "maxVisits": visits or self.config["max_visits"],
+            "maxVisits": visits,
             "komi": analysis_node.komi,
             "boardXSize": size_x,
             "boardYSize": size_y,

game.py

@@ -24,7 +24,7 @@ class Game:
 
     DEFAULT_PROPERTIES = {"GM": 1, "FF": 4, "RU": "JP", "AP": "KaTrain:https://github.com/sanderland/katrain"}
 
-    def __init__(self, katrain, engine: Union[Dict, KataGoEngine], config: Dict, move_tree: GameNode = None):
+    def __init__(self, katrain, engine: Union[Dict, KataGoEngine], config: Dict, move_tree: GameNode = None, analyze_fast=False):
         self.katrain = katrain
         if isinstance(engine, KataGoEngine):
             engine = {"B": engine, "W": engine}
@@ -45,11 +45,11 @@ class Game:
 
         self.current_node = self.root
         self._init_chains()
-        threading.Thread(target=lambda: self.analyze_all_nodes(-1_000_000), daemon=True).start()  # return faster, but bypass Kivy Clock
+        threading.Thread(target=lambda: self.analyze_all_nodes(-1_000_000, analyze_fast=analyze_fast), daemon=True).start()  # return faster, but bypass Kivy Clock
 
-    def analyze_all_nodes(self, priority=0):
+    def analyze_all_nodes(self, priority=0, analyze_fast=False):
         for node in self.root.nodes_in_tree:
-            node.analyze(self.engines[node.next_player], priority=priority)
+            node.analyze(self.engines[node.next_player], priority=priority, analyze_fast=analyze_fast)
 
     # -- move tree functions --
     def _init_chains(self):
@@ -199,7 +199,7 @@ class Game:
     def __repr__(self):
         return "\n".join("".join(Move.PLAYERS[self.chains[c][0].player] if c >= 0 else "-" for c in line) for line in self.board) + f"\ncaptures: {self.prisoner_count}"
 
-    def write_sgf(self, path=None, trainer_config={}, save_feedback=(True, True, True, True, True, True)):
+    def write_sgf(self, path=None, trainer_config={}, save_feedback=(True,), eval_thresholds=(0,)):
         black, white = self.root.get_property("PB"), self.root.get_property("PW")
         black = re.sub(r"['<>:\"/\\|?*]", "", black or "Black")
         white = re.sub(r"['<>:\"/\\|?*]", "", white or "White")
@@ -208,8 +208,7 @@ class Game:
         os.makedirs(os.path.dirname(file_name), exist_ok=True)
 
         show_dots_for = {p: trainer_config.get("eval_show_ai", True) or "ai" not in self.katrain.controls.player_mode(p) for p in Move.PLAYERS}
-        thresholds = self.katrain.config("trainer/eval_thresholds")
-        sgf = self.root.sgf(save_comments_player=show_dots_for, save_comments_class=save_feedback, eval_thresholds=thresholds)
+        sgf = self.root.sgf(save_comments_player=show_dots_for, save_comments_class=save_feedback, eval_thresholds=eval_thresholds)
         with open(file_name, "w") as f:
             f.write(sgf)
         return f"SGF with analysis written to {file_name}"

game_node.py

@@ -25,18 +25,30 @@ class GameNode(SGFNode):
             show_class = save_comments_class[evaluation_class(self.points_lost, eval_thresholds)]
         else:
             show_class = False
-        if save_comments_player.get(self.player, False) and show_class:
-            best_sq = []
+        if save_comments_player.get(self.player, False) and show_class and self.analysis_ready:
+            candidate_moves = self.candidate_moves
+            top_x = Move.from_gtp(candidate_moves[0]["move"]).sgf(self.board_size)
+            best_sq = [Move.from_gtp(d["move"]).sgf(self.board_size) for d in candidate_moves[1:] if d["pointsLost"] <= 0.5]
             if best_sq and "SQ" not in properties:
                 properties["SQ"] = best_sq
+            if top_x and "MA" not in properties:
+                properties["MA"] = [top_x]
             comment = self.comment(sgf=True)
             if comment:
                 properties["C"] = [properties.get("C", "") + comment]
+        if self.is_root:
+            properties["C"] = [
+                "Moves marked 'X' indicate the top move according to KataGo, those with a square are all moves which lose at most 0.5 points.\n"
+                + properties.get("C", "")
+                + "\nSGF with review generated by KaTrain."
+            ]
         return properties
 
     # various analysis functions
-    def analyze(self, engine, priority=0, visits=None, time_limit=True, refine_move=None):
-        engine.request_analysis(self, lambda result: self.set_analysis(result, refine_move), priority=priority, visits=visits, time_limit=time_limit, next_move=refine_move)
+    def analyze(self, engine, priority=0, visits=None, time_limit=True, refine_move=None, analyze_fast=False):
+        engine.request_analysis(
+            self, lambda result: self.set_analysis(result, refine_move), priority=priority, visits=visits, analyze_fast=analyze_fast, time_limit=time_limit, next_move=refine_move
+        )
 
     def update_move_analysis(self, move_analysis, move_gtp):
         cur = self.analysis["moves"].get(move_gtp)

gui/badukpan.py

@@ -255,7 +255,11 @@ class BadukPanWidget(Widget):
                     m = child_node.single_move
                     if m and m.coords is not None:
                         undo_coords.add(m.coords)
-                        evalcol = (*self.eval_color(points_lost), alpha) if points_lost is not None else None
+                        if points_lost is None:
+                            evalcol = None
+                        else:
+                            evalcol = self.eval_color(points_lost)
+                            evalcol[3] = alpha
                         scale = self.ui_config["child_scale"]
                         self.draw_stone(m.coords[0], m.coords[1], (*stone_color[m.player][:3], alpha), None, None, evalcol, evalscale=scale, scale=scale)
 

gui/controls.py

@@ -12,7 +12,7 @@ class Controls(BoxLayout):
 
     def set_status(self, msg, at_node=None):
         self.status = msg
-        self.status_node = at_node or self.katrain.game and self.katrain.game.current_node
+        self.status_node = at_node or self.katrain and self.katrain.game and self.katrain.game.current_node
         self.status_label.text = msg
         self.update_evaluation()
 
@@ -40,7 +40,7 @@ class Controls(BoxLayout):
     # handles showing completed analysis and score graph
     def update_evaluation(self):
         katrain = self.katrain
-        current_node = katrain.game and katrain.game.current_node
+        current_node = katrain and katrain.game and katrain.game.current_node
 
         if current_node is not self.status_node and not (self.status is not None and self.status_node is None and current_node.is_root):  # startup errors on root
             self.status_label.text = ""
@@ -79,7 +79,7 @@ class Controls(BoxLayout):
         config_popup.open()
 
     def configure_teacher(self):
-        config_popup = Popup(title="Edit Teacher Settings", size_hint=(0.6, 0.8))
+        config_popup = Popup(title="Edit Teacher Settings", size_hint=(0.7, 0.8))
         popup_contents = ConfigTeacherPopup(self.katrain, config_popup)
         config_popup.add_widget(popup_contents)
         config_popup.open()

gui/kivyutils.py

@@ -213,6 +213,9 @@ class ScoreGraph(Label):
     dot_pos = ListProperty([0, 0])
     highlighted_index = NumericProperty(None)
     y_scale = NumericProperty(4)
+    marginx = NumericProperty(0.015)
+    marginy = NumericProperty(0.01)
+    highlight_size = NumericProperty(5)
 
     def __init__(self, **kwargs):
         super().__init__(**kwargs)
@@ -235,19 +238,21 @@ class ScoreGraph(Label):
 
             self.y_scale = math.ceil(max(4, max(-val_range[0], val_range[1])) / 2) * 2
 
-            xscale = self.width * 0.9 / max(len(values) - 1, 15)
+            xscale = self.width * (1 - 2 * self.marginx) / max(len(values) - 1, 15)
             available_height = self.height * (1 - 2 * self.marginy)
-            line_points = [[self.pos[0] + self.marginx * self.width + i * xscale, self.pos[1] + available_height / 2 * (1 + val / self.y_scale)] for i, val in enumerate(values)]
+            line_points = [
+                [self.pos[0] + self.marginx * self.width + i * xscale, self.pos[1] + self.height / 2 + available_height / 2 * (val / self.y_scale)] for i, val in enumerate(values)
+            ]
             self.line_points = sum(line_points, [])
 
             if self.highlighted_index is not None:
                 self.highlighted_index = min(self.highlighted_index, len(values) - 1)
                 dot_point = line_points[self.highlighted_index]
                 if math.isnan(dot_point[1]):
-                    dot_point[1] = self.pos[1] + available_height / 2 * (1 + (nn_values or [0])[-1] / self.y_scale)
+                    dot_point[1] = self.pos[1] + self.height / 2 + available_height / 2 * ((nn_values or [0])[-1] / self.y_scale)
                 self.dot_pos = [c - self.highlight_size / 2 for c in dot_point]
 
-    #            print("Graph updated to ", len(line_points), "points, hl=", self.highlighted_index, self.dot_pos)
+    #                print("Graph updated to ", len(line_points), "points, hl=", self.highlighted_index, self.dot_pos)
 
     def update_value(self, node):
         self.highlighted_index = index = node.depth

gui/popups.py

@@ -169,7 +169,8 @@ class ConfigPopup(QuickConfigGui):
                     old_engine.shutdown(finish=True)
                 else:
                     self.katrain.game.analyze_all_nodes()  # old engine was broken, so make sure we redo any failures
-                Clock.schedule_once(restart_engine, 0)
+
+            Clock.schedule_once(restart_engine, 0)
 
         self.katrain.debug_level = self.config["debug"]["level"]
         self.katrain.update_state(redraw_board=True)
@@ -235,6 +236,8 @@ class ConfigAIPopup(QuickConfigGui):
 class ConfigTeacherPopup(QuickConfigGui):
     def __init__(self, katrain, popup, **kwargs):
         self.settings = katrain.config("trainer")
+        self.sgf_settings = katrain.config("sgf")
+        print(self.sgf_settings, katrain.config("sgf"))
         self.ui_settings = katrain.config("board_ui")
         super().__init__(katrain, popup, self.settings, **kwargs)
         Clock.schedule_once(self._build, 0)
@@ -245,18 +248,18 @@ class ConfigTeacherPopup(QuickConfigGui):
         thresholds = self.settings["eval_thresholds"]
         undos = self.settings["num_undo_prompts"]
         colors = self.ui_settings["eval_colors"]
-        thrbox = GridLayout(spacing=1, padding=2, cols=4, rows=len(thresholds) + 1)
+        thrbox = GridLayout(spacing=1, padding=2, cols=5, rows=len(thresholds) + 1)
         thrbox.add_widget(ScaledLightLabel(text="Point loss greater than", bold=True))
         thrbox.add_widget(ScaledLightLabel(text="Gives this many undos", bold=True))
         thrbox.add_widget(ScaledLightLabel(text="Color (fixed)", bold=True))
         thrbox.add_widget(ScaledLightLabel(text="Show dots", bold=True))
-
+        thrbox.add_widget(ScaledLightLabel(text="Save in SGF", bold=True))
         for i, (thr, undos, color) in enumerate(zip(thresholds, undos, colors)):
             thrbox.add_widget(LabelledFloatInput(text=str(thr), input_property=f"eval_thresholds::{i}"))
             thrbox.add_widget(LabelledFloatInput(text=str(undos), input_property=f"num_undo_prompts::{i}"))
             thrbox.add_widget(BackgroundLabel(background=color[:3]))
             thrbox.add_widget(LabelledCheckBox(text=str(color[3] == 1), input_property=f"alpha::{i}"))
-
+            thrbox.add_widget(LabelledCheckBox(size_hint=(0.5, 1), text=str(self.sgf_settings["save_feedback"][i]), input_property=f"save_feedback::{i}"))
         self.add_widget(thrbox)
 
         xsettings = BoxLayout(size_hint=(1, 0.15), spacing=2)
@@ -264,7 +267,7 @@ class ConfigTeacherPopup(QuickConfigGui):
         xsettings.add_widget(LabelledIntInput(size_hint=(0.5, 1), text=str(self.settings["eval_off_show_last"]), input_property="eval_off_show_last"))
         self.add_widget(xsettings)
         xsettings = BoxLayout(size_hint=(1, 0.15), spacing=2)
-        xsettings.add_widget(ScaledLightLabel(text="Show dots for AI players"))
+        xsettings.add_widget(ScaledLightLabel(text="Show dots/SGF comments for AI players"))
         xsettings.add_widget(LabelledCheckBox(size_hint=(0.5, 1), text=str(self.settings["eval_show_ai"]), input_property="eval_show_ai"))
         self.add_widget(xsettings)
 
@@ -281,15 +284,20 @@ class ConfigTeacherPopup(QuickConfigGui):
                 if "::" in k:
                     k1, i = k.split("::")
                     i = int(i)
-                    if "alpha" not in k1:
-                        if self.settings[k1][i] != v:
-                            self.settings[k1][i] = v
-                            self.katrain.log(f"Updating setting {k1}[{i}] = {v}", OUTPUT_DEBUG)
-                    else:
+                    if "alpha" in k1:
                         v = 1.0 if v else 0.0
                         if self.ui_settings["eval_colors"][i][3] != v:
                             self.katrain.log(f"Updating alpha {i} = {v}", OUTPUT_DEBUG)
                             self.ui_settings["eval_colors"][i][3] = v
+                    elif "save_feedback" in k1:
+                        if self.sgf_settings[k1][i] != v:
+                            self.sgf_settings[k1][i] = v
+                            self.katrain.log(f"Updating setting sgf/{k1}[{i}] = {v}", OUTPUT_DEBUG)
+
+                    else:
+                        if self.settings[k1][i] != v:
+                            self.settings[k1][i] = v
+                            self.katrain.log(f"Updating setting trainer/{k1}[{i}] = {v}", OUTPUT_DEBUG)
                 else:
                     if self.settings[k] != v:
                         self.settings[k] = v

katrain.kv

@@ -67,8 +67,8 @@
 <ToggleButtonContainer>:
 
 <LabelledTextInput>:
-    padding: 3, 0.3*self.size[1]
-    font_size: 0.33*self.size[1]
+    padding: 3, 0.27*self.size[1]
+    font_size: 0.4*self.size[1]
 
 <LabelledIntInput>:
     font_size: 0.5*self.size[1]
@@ -151,7 +151,6 @@
 	size: self.texture_size
 
 <ClickableLabel@DarkLabel+ButtonBehavior>:
-    font_size: dp(12)
     color: BLACK
 	halign: 'center'
 	valign: 'center'
@@ -161,21 +160,22 @@
 	orientation: 'horizontal'
 	text: ''
 	label: ''
-	ClickableLabel:
-		size_hint: 0.6,None
-		height: self.font_size * 1.5
+    ClickableLabel:
+		size_hint: 0.6,1
 		id: desc
 		text: root.label
-		font_size: dp(20)
-		on_press: value.opacity =  1 if  value.opacity==0 else 0
+		font_size: self.size[1] * 0.8
+		on_press:
+		    value.opacity =  1 if  value.opacity==0 else 0
 	DarkLabel:
-		size_hint: 0.4,None
+		size_hint: 0.4,1
 		text: root.text
-		height: desc.height
 		font_size: desc.font_size
 		id: value
 		bold: True
 
+
+
 <BaseCircleWithText>:
     text: ''
     color: WHITE
@@ -216,9 +216,6 @@
     size: self.texture_size
 
 <ScoreGraph>:
-    highlight_size: 5
-    marginx: 0.05
-    marginy: 0.01
     whalf: root.pos[1]+self.height*0.25*(1 + 2 * root.marginy)
     bhalf: root.pos[1]+self.height*0.75*(1 - 2/3 * root.marginy)
     mid: root.pos[1]+self.height*0.5
@@ -227,19 +224,19 @@
         Color:
             rgba: GREY
         Line:
-            points: [root.pos[0]+self.width*root.marginx,root.whalf,root.pos[0]+self.width*0.95,root.whalf]
+            points: [root.pos[0]+self.width*root.marginx,root.whalf,root.pos[0]+self.width*(1-root.marginx),root.whalf]
             width: 1.0
             dashes: [1,2]
         Line:
-            points: [root.pos[0]+self.width*root.marginx,root.mid,root.pos[0]+self.width*0.95,root.mid]
+            points: [root.pos[0]+self.width*root.marginx,root.mid,root.pos[0]+self.width*(1-root.marginx),root.mid]
             width: 1.0
             dashes: [4,4]
         Line:
-            points: [root.pos[0]+self.width*root.marginx,root.bhalf,root.pos[0]+self.width*0.95,root.bhalf]
+            points: [root.pos[0]+self.width*root.marginx,root.bhalf,root.pos[0]+self.width*(1-root.marginx),root.bhalf]
             width: 1.0
             dashes: [1,2]
         Line:
-            rectangle: [root.pos[0] + root.marginx * self.width, root.pos[1] + root.marginy*self.height, root.width*0.9, root.height*(1-2*root.marginy)]
+            rectangle: [root.pos[0] + root.marginx * self.width, root.pos[1] + root.marginy*self.height, self.width*(1-2*root.marginx), root.height*(1-2*root.marginy)]
     canvas.after:
         Color:
             rgba: BLACK
@@ -357,6 +354,11 @@
             Ellipse:
                 pos: self.pos[0] + self.width/2 - self.height/5, self.pos[1]  + self.height/2 - self.height/5
                 size: self.height/2.5,self.height/2.5
+            Color:
+                rgba: BLACK
+            Line:
+                circle: self.pos[0] + self.width/2, self.pos[1] + self.height/2, self.height/5 - 1
+                width: 1.1
     Label:
         size_hint: 0.025, 1
 
@@ -374,11 +376,11 @@
         Color:
             rgba: [*[(c+1)/2 for c in root.border_color[:3]], root.border_color[3]]
         Line:
-            rectangle: [self.pos[0]+1,self.pos[1]+2,self.width-3,self.height-4]
+            rectangle: [self.pos[0]+1,self.pos[1]+2,self.width-3,self.height-3]
             width:1
     Label:
         padding: 5, 5
-        font_size: dp(20)
+        font_size: dp(18)
         color: BLACK
         size_hint_y: None
         height: self.texture_size[1]
@@ -500,16 +502,20 @@
             ScoreGraph:
                 id: graph
                 size_hint: 1,0.7
-            CensorableLabel:
-                id: win_rate
-                size_hint: 1, 0.15
-                label: 'Win rate'
-                text: ''
-            CensorableLabel:
-                id: score
-                size_hint: 1, 0.15
-                label: 'Score'
-                text: ''
+            BoxLayout:
+                orientation: 'vertical'
+                size_hint: 1,0.3
+                spacing: 3
+                CensorableLabel:
+                    id: win_rate
+                    size_hint: 1, 0.5
+                    label: 'Win rate'
+                    text: ''
+                CensorableLabel:
+                    id: score
+                    size_hint: 1, 0.5
+                    label: 'Score'
+                    text: ''
         BoxLayout: # ----------- play -----------
             orientation: 'vertical'
             size_hint: 1,1
@@ -563,7 +569,7 @@
                 StyledButton:
                     text: 'Configure Teacher'
                     on_press: root.configure_teacher()
-                    size_hint: 1.33, 1
+                    size_hint: 1.425, 1
                 StyledButton:
                     text: 'Configure AIs'
                     on_press: root.configure_ais()
@@ -572,7 +578,7 @@
                 size_hint: 1,0.2
     CensorableLabel:
         id: score_change
-        size_hint: 1, 0.0225
+        size_hint: 1, 0.025
         label: 'Score change'
         text: ''
     BoxLayout:

katrain.py

@@ -111,6 +111,19 @@ class KaTrainGui(BoxLayout):
         self.board_controls.mid_circles_container.add_widget(top)
         self.board_controls.black_prisoners.text = str(prisoners[1])
         self.board_controls.white_prisoners.text = str(prisoners[0])
+
+        # update engine status dot
+        if not self.engine or not self.engine.katago_process or self.engine.katago_process.poll() is not None:
+            self.board_controls.engine_status_col = self.config("board_ui/engine_down_col")
+        elif len(self.engine.queries) >= 4:
+            self.board_controls.engine_status_col = self.config("board_ui/engine_busy_col")
+        elif len(self.engine.queries) >= 2:
+            self.board_controls.engine_status_col = self.config("board_ui/engine_little_busy_col")
+        elif len(self.engine.queries) == 0:
+            self.board_controls.engine_status_col = self.config("board_ui/engine_ready_col")
+        else:
+            self.board_controls.engine_status_col = self.config("board_ui/engine_almost_done_col")
+        # redraw
         if redraw_board:
             Clock.schedule_once(self.board_gui.draw_board, -1)  # main thread needs to do this
         Clock.schedule_once(self.board_gui.draw_board_contents, -1)
@@ -137,9 +150,9 @@ class KaTrainGui(BoxLayout):
         if self.game:
             self.message_queue.put([self.game.game_id, message, *args])
 
-    def _do_new_game(self, move_tree=None):
+    def _do_new_game(self, move_tree=None, analyze_fast=False):
         self.engine.on_new_game()  # clear queries
-        self.game = Game(self, self.engine, self.config("game"), move_tree=move_tree)
+        self.game = Game(self, self.engine, self.config("game"), move_tree=move_tree, analyze_fast=analyze_fast)
         self.controls.select_mode("analyze" if move_tree and len(move_tree.nodes_in_tree) > 1 else "play")
         self.controls.graph.initialize_from_game(self.game.root)
         self.update_state(redraw_board=True)
@@ -148,7 +161,6 @@ class KaTrainGui(BoxLayout):
         if node is None or self.game.current_node == node:
             mode = self.controls.ai_mode(self.game.current_node.next_player)
             settings = self.config(f"ai/{mode}")
-            print(mode, settings)
             if settings:
                 ai_move(self.game, mode, settings)
 
@@ -174,7 +186,7 @@ class KaTrainGui(BoxLayout):
         fileselect_popup = Popup(title="Double Click SGF file to analyze", size_hint=(0.8, 0.8))
         popup_contents = LoadSGFPopup()
         fileselect_popup.add_widget(popup_contents)
-        popup_contents.filesel.path = os.path.expanduser(self.config("files/sgf_load"))
+        popup_contents.filesel.path = os.path.expanduser(self.config("sgf/sgf_load"))
 
         def readfile(files, _mouse):
             fileselect_popup.dismiss()
@@ -183,7 +195,7 @@ class KaTrainGui(BoxLayout):
             except ParseError as e:
                 self.log(f"Failed to load SGF. Parse Error: {e}", OUTPUT_ERROR)
                 return
-            self._do_new_game(move_tree=move_tree)
+            self._do_new_game(move_tree=move_tree, analyze_fast=fileselect_popup.fast.active)
 
         popup_contents.filesel.on_submit = readfile
         fileselect_popup.open()
@@ -207,7 +219,12 @@ class KaTrainGui(BoxLayout):
                 self.game.root.set_property(
                     f"P{pl}", f"AI {self.controls.ai_mode(pl)} (KataGo { os.path.splitext(model_file)[0]})" if "ai" in self.controls.player_mode(pl) else "Player"
                 )
-        msg = self.game.write_sgf(self.config("files/sgf_save"), trainer_config=self.config("trainer"), save_feedback=self.config("sgf/save_feedback"))
+        msg = self.game.write_sgf(
+            self.config("sgf/sgf_save"),
+            trainer_config=self.config("trainer"),
+            save_feedback=self.config("sgf/save_feedback"),
+            eval_thresholds=self.config("trainer/eval_thresholds"),
+        )
         self.log(msg, OUTPUT_INFO)
         self.controls.set_status(msg)
 
@@ -264,7 +281,7 @@ class KaTrainGui(BoxLayout):
                 self.controls.set_status(f"Failed to imported game from clipboard: {e}\nClipboard contents: {clipboard[:50]}...")
                 return
             move_tree.nodes_from_root[-1].analyze(self.engine)  # speed up result for looking at end of game
-            self._do_new_game(move_tree=move_tree)
+            self._do_new_game(move_tree=move_tree, analyze_fast=True)
             self("redo", 999)
             self.log("Imported game from clipboard.", OUTPUT_INFO)
         return True

selfplay.py

@@ -18,7 +18,7 @@ DB_FILENAME = "ai_performance.pickle"
 
 class Logger:
     def log(self, msg, level):
-        if level <= OUTPUT_DEBUG:
+        if level <= OUTPUT_INFO:
             print(msg)
         if level <= OUTPUT_ERROR:
             print(msg, file=sys.stderr)
@@ -147,6 +147,31 @@ test_ais = [
     AI("P:Weighted", {"pick_override": 1.0}),
 ]
 
+test_ais = [
+    AI("Policy", {}),
+    AI("P:Noise", {"noise_strength": 0.8}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.0, "weaken_fac": 1}),
+    AI("P:Weighted", {"pick_override": 0.95, "lower_bound": 0.0, "weaken_fac": 1}),
+    AI("P:Weighted", {"pick_override": 0.9, "lower_bound": 0.0, "weaken_fac": 1}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.0, "weaken_fac": 0.5}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.0, "weaken_fac": 1}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.0, "weaken_fac": 1.5}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.0, "weaken_fac": 2}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.01, "weaken_fac": 0.5}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.01, "weaken_fac": 1}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.01, "weaken_fac": 1.5}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.01, "weaken_fac": 2}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.001, "weaken_fac": 0.5}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.001, "weaken_fac": 1}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.001, "weaken_fac": 1.5}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.001, "weaken_fac": 2}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.005, "weaken_fac": 0.5}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.005, "weaken_fac": 1}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.005, "weaken_fac": 1.5}),
+    AI("P:Weighted", {"pick_override": 1.0, "lower_bound": 0.005, "weaken_fac": 2}),
+    AI("P:Pick", {"pick_frac": 0.5, "pick_n": 0}),
+]
+
 
 # test_ais = [
 #    AI("Policy", {}),
@@ -161,7 +186,7 @@ test_ais = [
 for ai in test_ais:
     add_ai(ai)
 
-N_GAMES = 5
+N_GAMES = 3
 
 ais_to_test = retrieve_ais(test_ais)
 # ais_to_test = ai_database

sgf_parser.py

@@ -82,7 +82,7 @@ class SGFNode:
         sys.setrecursionlimit(max(sys.getrecursionlimit(), 3 * 29 * 29))  # thanks to lightvector for causing stack overflows
         sgf_str = "".join([prop + "".join(f"[{v}]" for v in values) for prop, values in self.sgf_properties(**xargs).items() if values])
         if self.children:
-            children = [c.sgf() for c in self.children]
+            children = [c.sgf(**xargs) for c in self.children]
             if len(children) == 1:
                 sgf_str += ";" + children[0]
             else: