LCZero

[Владимир001]

Match: PR963 (betamcts) vs baseline
LC0 version (both): deeabd7
LC0 options (both): threads=1, minibatch-size=192, cpuct=1.32, cpuct-at-root=1.9, cpuct-factor=2.815, cpuct-factor-at-root=2.815, fpu-value=0.23, fpu-strategy-at-root=absolute, policy-softmax-temp=1.4, moves-left-max-effect=0.2, moves-left-threshold=0, moves-left-slope=0.007, moves-left-scaled-factor=0.15, moves-left-quadratic-factor=0.85
LC0 options (betamcts): betamcts-level=2, betamcts-trust=4.2, betamcts-prior=0.04, lcb-percentile=0.2, betamcts-update-interval=10, april-factor=0, april-factor-parent=0
LC0 options (baseline): betamcts-level=0, betamcts-trust=0, betamcts-prior=0, lcb-percentile=0, betamcts-update-interval=10, april-factor=0, april-factor-parent=0
Time control: 800 nodes/move
Net: 703810
Book: custom-3k (1-3 ply)
Hardware: gtx970 + i5-4570
Tablebases: syzygy 5-man
Adjudication: -draw movenumber=50 movecount=5 score=10 -resign movecount=5 score=700 twosided=true
Software: cutechess-cli
Comment: using 800npm tuned values... time to merge this for training? @Naphthalin

   # PLAYER                :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W     D    L  D(%)
   1 lc0-703810-betamcts   :    33.0    9.1  1094.0    2000    55     100  443  1302  255    65
   2 lc0-703810            :     0.0   ----   906.0    2000    45     ---  255  1302  443    65

White advantage = 2.48 +/- 4.68
Draw rate (equal opponents) = 65.96 % +/- 1.09

https://pixeldrain.com/u/19JtLYSi

[Омегазеро]

Новая сеть https://storage.lczero.org/files/net...ontrib/J92-300

[Владимир001]

Разработка методов обучения с подкреплением, которые находят хорошую политику с как можно меньшим количеством образцов, является ключевой целью как эмпирических, так и теоретических исследований..

https://blog.ml.cmu.edu/2019/08/16/p...ment-learning/

[Владимир001]

FINAL RESULT 100/100 games

Stockfish won the SuFi comprehensively with the SuFi 19 opening book.

Using a balanced 8 ply book, the match was drawn.

Openings do seem to make a difference.

https://www.twitch.tv/navratil25

Stockfish Sufi
Lc0-Sufi

8 ply balanced book
60’ + 60”

   # PLAYER               :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W    D    L  D(%)
   1 stockfish-Sufi 19    :       0      7    50.0     100  50.0      50    2   96    2  96.0
   2 Lc0-Sufi             :       0      7    50.0     100  50.0     ---    2   96    2  96.0

White advantage = 0.00 +/- 0.00
Draw rate (equal opponents) = 96.00 % +/- 2.33

MEDIANS

Name                     Games    Depth       NPS           Nodes
-------------------------------------------------------------------------------------
stockfish-Sufi 19    :      100     52        32289968   2862175708
Lc0-Sufi             :      100     18           40882      8797314

NEXT MATCH: Bookless Openings

https://pixeldrain.com/u/VSw9wb45

[Владимир001]

http://www.fastgm.de/16-60-0.6.html

https://www.sp-cc.de/nn-vs-sf-testing.htm

[Владимир001]

Развернуть для просмотра

Match: combined results (314 positions) of ERET test + Vesely 203 positions testset
time control: analysis with time limit of 60s per move/position
AB engine options: threads=4, hash=4096 MB
Lc0-version: v0.26.2 + v0.26.3
LC0-options: -Backend=cudnn -NNcache=3000000 -MaxCcollisionEvents=256
MLH-settings S19 SuFi: -MovesLeftMaxEffect=0.2 -MovesLeftSlope=0.004 -MovesLeftThreshold=0.0 -MovesQuadraticFactor=0.0 -MovesScaledFactor=1.0
hardware: notebook with mobile i7-7700HQ 4x2.8GHz, mobile Geforce GTX 1050 TI 4GB
software: Arena GUI, "Automatic Analysis" feature with EPD files.
Tablebases: Syzygy 5-men on SSD
Comments: J92-300 added. For J92 up to 280 so far those results fit nicely to the results of the test gauntlet against SF Dev at 5min+5s TC (300/290 not tested there yet).

A) CPU engines (mobile i7-7700HQ 4x2.8GHz):
Cfish 221020 NNUE 4CPU          :  280/314 = 89.2%   *net: nn-eba324f53044.nnue*
CorChess NNUE 1.3_211020 4CPU   :  279/314 = 88.9%   *net: nn-eba324f53044.nnue*

B) GPU engines (mobile Geforce GTX 1050 TI 4GB):
Lc0.net.JH.J92-280  (30x384)    :  220/314 = 70.0%
Lc0.net.JH.J92-260  (30x384)    :  218/314 = 69.4%
Lc0.net.JH.J92-270  (30x384)    :  217/314 = 69.1%
Lc0.net.JH.J92-290  (30x384)    :  216/314 = 68.8%
Lc0.net.JH.J92-210  (30x384)    :  216/314 = 68.8%
Lc0.net.JH.J92-240  (30x384)    :  215/314 = 68.5%
Lc0-0.22.0-dev.Leelenstein 14.3 :  215/314 = 68.5%
Lc0.net.JH.J92-300  (30x384)    :  214/314 = 68.2%   <== NEW
Lc0.net.JH.J92-190  (30x384)    :  214/314 = 68.2%   ** TCEC S19 SuFi net **
Lc0.net.65536                   :  213/314 = 67.8%
Lc0.net.65162                   :  213/314 = 67.8%
Lc0.net.65411                   :  205/314 = 65.3%

[свернуть]

[Владимир001]

Развернуть для просмотра

Match: J92-300 vs stockfish 12 in fixed nodes
LC0 version: 0.26.2, built with cuda11 + cudnn7
LC0 options: cudnn-fp16, 1 thread, default for the rest
SF options: 1 thread, 1GB Hash.
Time control: Leela: 1 kn/move, SF 12: 400Kn/move
Hardware: RTX 2070, i7-7700
Book: Balsa 2724 openings
Tablebases: 6-man TB for both.
Adjudication: 6-man TB, -draw movenumber=50 movecount=5 score=8 -resign movecount=5 score=1000
Software:cutechess-cli
Comments: J92-300 got -5 elo vs SF 12, quite low result here.

Leela: 1 thread, 1Kn/move
SF 12: 1 thread, 400Kn/move

Balsa 2724

   # PLAYER                :  RATING  ERROR   POINTS  PLAYED   (%)  CFS(%)      W      D      L
-  1 Stockfish_12.400k     :       0   ----  27640.0   54480  50.7      57  10812  33656  10012
   2 lc0.net.J92-270.1k    :      -1      6   2720.0    5448  49.9      68   1063   3314   1071
   3 lc0.net.J92-290.1k    :      -2      6   2705.5    5448  49.7      53   1012   3387   1049
   4 lc0.net.J92-260.1k    :      -3      6   2703.0    5448  49.6      54   1004   3398   1046
   5 lc0.net.J92-280.1k    :      -3      6   2699.5    5448  49.6      55   1045   3309   1094
   6 lc0.net.J92-190.1k    :      -4      5   2696.0    5448  49.5      54    996   3400   1052
   7 lc0.net.J92-250.1k    :      -4      6   2693.0    5448  49.4      61   1015   3356   1077
   8 lc0.net.J92-230.1k    :      -5      6   2684.0    5448  49.3      51    980   3408   1060
+  9 lc0.net.J92-300.1k    :      -5      6   2683.5    5448  49.3      80   1009   3349   1090
  10 lc0.net.J92-220.1k    :      -9      6   2656.0    5448  48.8      96    978   3356   1114
  11 lc0.net.J92-100.1k    :     -16      6   2599.5    5448  47.7     ---    910   3379   1159

[свернуть]

https://pixeldrain.com/u/8rsnj5KR

[Владимир001]

Развернуть для просмотра

Hardware: i7-8770K CPU @ 3.70GHz, 6 cores (12 Threads), 32GB, Gpu - RTX 2080 | Lc0 vs SF Dev Scaling
Gauntlet: SF-dev vs. LC0 30x384 net
Network IDs: Lc0 J92-270
LC0-version: v0.26.3
SF-version: SF-dev 5th Oct (NNUE hybrid)
Software: Cutechess-CLI
Time control: Lc0 1k vs SF NNUE 350k | Lc0 2k vs SF NNUE 700k | Lc0 4k vs SF NNUE 1400k | Lc0 10k vs SF NNUE 3500k | Lc0 20k vs SF NNUE 7000k | Lc0 40k vs SF NNUE 14000k per move. | Lc0 with default settings and 1 thread, SF NNUE with default settings and 4 threads. Check attached bat files for more information.
Book: 8moves_v3.pgn (SF Regression book at LTC)
Tablebases: 6-men Synergy WDL with TB Adjudication
Adjudication: -draw movenumber=50 movecount=5 score=8 -resign movecount=5 score=1000
Ordo: ordo-win64.exe -Q -N "1,1" -D -a 0 -A "Stockfish_Dev4T" -W -n4 -s5000 -p games.pgn -U "0,1,2,3,4,5,6,7,8,9,10" -o rating.txt
Comment: You can take a look at Lc0 Jh 270 vs SF NNUE (5th Oct) Scaling. At your perusal.

Lc0 1k vs SF NNUE 350k

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)     W     D     L  D(%)
   1 Lc0 Jh 270          :     6.5    4.2  5091.5   10000  50.9     100  2034  6115  1851  61.1
   2 Stockfish_Dev_4T    :     0.0   ----  4908.5   10000  49.1     ---  1851  6115  2034  61.1

White advantage = 39.97 +/- 2.18
Draw rate (equal opponents) = 62.23 % +/- 0.49

Lc0 2k vs SF NNUE 700k

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)     W     D     L  D(%)
   1 Stockfish_Dev_4T    :     0.0   ----  3025.5    6000  50.4      87  1029  3993   978  66.5
   2 Lc0 Jh 270          :    -3.0    5.2  2974.5    6000  49.6     ---   978  3993  1029  66.5

White advantage = 39.84 +/- 2.65
Draw rate (equal opponents) = 67.90 % +/- 0.61

Lc0 4k vs SF NNUE 1400k

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W     D    L  D(%)
   1 Stockfish_Dev_4T    :     0.0   ----  2029.5    4000  50.7      96  594  2871  535  71.8
   2 Lc0 Jh 270          :    -5.2    5.8  1970.5    4000  49.3     ---  535  2871  594  71.8

White advantage = 36.33 +/- 2.98
Draw rate (equal opponents) = 73.24 % +/- 0.72

Lc0 10k vs SF NNUE 3500k

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W     D    L  D(%)
   1 Stockfish_Dev_4T    :     0.0   ----  1017.5    2000  50.9      96  234  1567  199  78.3
   2 Lc0 Jh 270          :    -6.2    7.1   982.5    2000  49.1     ---  199  1567  234  78.3

White advantage = 34.65 +/- 3.69
Draw rate (equal opponents) = 80.31 % +/- 0.93

Lc0 20k vs SF NNUE 7000k

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W     D    L  D(%)
   1 Stockfish_Dev_4T    :     0.0   ----  1013.0    2000  50.6      92  188  1650  162  82.5
   2 Lc0 Jh 270          :    -4.6    6.4   987.0    2000  49.4     ---  162  1650  188  82.5

White advantage = 33.05 +/- 3.28
Draw rate (equal opponents) = 84.86 % +/- 0.87

Lc0 40k vs SF NNUE 14000k

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W     D    L  D(%)
   1 Stockfish_Dev_4T    :     0.0   ----  1012.5    2000  50.6      93  151  1723  126  86.2
   2 Lc0 Jh 270          :    -4.4    5.7   987.5    2000  49.4     ---  126  1723  151  86.2

White advantage = 26.87 +/- 2.94
Draw rate (equal opponents) = 88.23 % +/- 0.79

[свернуть]

https://pixeldrain.com/u/B41LtnrY

https://pixeldrain.com/u/GfswvKBm

[Владимир001]

Цитата: Сообщение от Омегазеро

Новая сеть https://storage.lczero.org/files/net...ontrib/J92-300

J92-300 running now @ https://www.twitch.tv/VendableBee420

[svoitsl]

Почти 3000 само Эло!!!

[Владимир001]

С Дискорд:
Для всех: Мы приглашаем вас использовать и помочь нам протестировать новое приложение Lc0 (это приложение с открытым исходным кодом) для iOS (для iPhone, iPad и iWatch). Просто следуйте приведенным ниже ссылкам (сначала вы установите приложение TestFlight, а затем Lc0). Любая польза, помощь, предложение высоко ценятся:
https://testflight.apple.com/join/UxeYAGbb

[Омегазеро]

Тренировки продолжаются! Следующая сеть https://storage.lczero.org/files/net...ontrib/J92-310

[Владимир001]

------
Match Gauntlet
LC0-version: lc0-v0.26.3 cudnn-version
LC0 options: Standard option except Threads=1, NNCacheSize=0, MinibatchSize=1.
Time control: 1 Node/move
Hardware: i7-4790 with 16 GByte RAM + GTX1650
Speed: Not important for this type of testing.
Book: Chad-opening book with 100 000 different openings with the most popular human openings.
Book options: Opening order: Sequential. Play each opening twice. Play all opening in the book both for White and Black.
Tablebases: No Tablebases.
Adjudication: Only 5-men syzygy
Software: CuteChess 1.2.0 https://github.com/cutechess/cutechess/releases / Graphic tools is programmed by glbchess64, in Python programming tools.
Comment: This is result out of Ordo 1.2.6 https://github.com/michiguel/Ordo/releases/tag/v1.2.6
Ordo string: ordo.exe -Q -N 0 -D -a 0 -A "J92-145" -W -n4 -s500 -G -V -U "0,1,2,3,4,5,6,7,8,9,10" -j stats.txt -o rating.txt -p games.pgn
Results: pgn-file: https://ufile.io/7rqnyfr9
Explanation: Trying to find the best net in clean policy Elo.

   # PLAYER     :  RATING  ERROR    POINTS  PLAYED   (%)  CFS(%)      W      D      L  D(%)
   1 J92-290    :      10      1  102895.0  200000  51.4     100  66788  72214  60998  36.1
   2 J92-145    :       0      1   97105.0  200000  48.6     ---  60998  72214  66788  36.1

White advantage = 27.34 +/- 0.57
Draw rate (equal opponents) = 36.30 % +/- 0.10

My list until now (Elo against J92-145) 1 Node/move:
White advantage = 23.98 +/- 0.57
Draw rate (equal opponents) = 36.51 % +/- 0.10
J92-270:             17     Elo    (Test with 200k games)
J92-280:             11     Elo    (Test with 200k games)
J92-260:             11     Elo    (Test with 200k games)
J92-290:             10     Elo    (Test with 200k games)
J92-240:              8     Elo    (Test with 200k games)
J92-210:              5     Elo    (Test with 200k games)
J92-220:              3     Elo    (Test with 200k games)
J92-205:              3     Elo    (Test with 200k games)
J92-230:              2     Elo    (Test with 200k games)
J92-190:              1     Elo    (Test with 200k games)
J92-145:              0     Elo    (Reference)
J92-100:             -2     Elo    (Test with 200k games)

[Владимир001]

# PLAYER                :  RATING  ERROR    POINTS  PLAYED   (%)  CFS(%)      W       D      L  D(%)
   1 lc0.net.65297         :    30.2    3.3   10836.0   20000    54      58   5374   10924   3702    55
+  2 lc0.net.65373         :    29.7    3.2   10822.0   20000    54      54   5360   10924   3716    55
+  3 lc0.net.65375         :    29.4    3.4   10815.0   20000    54      52   5318   10994   3688    55
+  4 lc0.net.65377         :    29.3    3.2   10812.0   20000    54      57   5350   10924   3726    55
+  5 lc0.net.65378         :    28.9    3.3   10800.5   20000    54      76   5347   10907   3746    55
   6 lc0.net.65162         :    27.2    3.3   10754.0   20000    54      58   5255   10998   3747    55
+  7 lc0.net.65376         :    26.7    3.3   10740.0   20000    54      52   5244   10992   3764    55
+  8 lc0.net.65380         :    26.5    3.2   10736.0   20000    54      60   5283   10906   3811    55
+  9 lc0.net.65381         :    26.0    3.2   10720.0   20000    54      70   5281   10878   3841    54
+ 10 lc0.net.65379         :    24.7    3.2   10686.0   20000    53      50   5209   10954   3837    55
+ 11 lc0.net.65382         :    24.7    3.2   10685.5   20000    53      67   5199   10973   3828    55
+ 12 lc0.net.65374         :    23.7    3.2   10657.5   20000    53      90   5127   11061   3812    55
  13 lc0.net.T60B.7-135    :    20.8    3.3   10576.5   20000    53      63   4994   11165   3841    56
  14 lc0.net.T60B.7-105    :    20.0    3.2   10556.0   20000    53     100   4984   11144   3872    56
  15 lc0.net.64255         :     3.6    3.2   10100.0   20000    51      99   4463   11274   4263    56
- 16 Stockfish.dev         :     0.0   ----  139703.0  300000    47     ---  57194  165018  77788    55

White advantage = 57.43 +/- 0.43
Draw rate (equal opponents) = 57.06 % +/- 0.10

PGN: 10 files:

https://pixeldrain.com/l/N8g1do3C#item=0

[Владимир001]

Match: 730305 vs 703459
LC0 version: 0.26.3
LC0 options: threads=1, minibatch-size=192
MLH options: moves-left-max-effect=0.2, moves-left-threshold=0, moves-left-slope=0.004, moves-left-scaled-factor=1, moves-left-quadratic-factor=0
Time control: 60s+1s
Book: custom-3k (1-3 ply)
Hardware: gtx970 + i5-4570
Tablebases: syzygy 5-man
Adjudication: -draw movenumber=50 movecount=5 score=10 -resign movecount=5 score=700 twosided=true
Software: cutechess-cli
Comment: recent t73 net vs last t70 net of same LR

   # PLAYER        :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W    D    L  D(%)
   1 lc0-703459    :     0.0   ----    53.0     100    53      90   13   80    7    80
   2 lc0-730305    :   -21.1   31.8    47.0     100    47     ---    7   80   13    80

White advantage = 7.04 +/- 15.95
Draw rate (equal opponents) = 80.82 % +/- 4.27

https://pixeldrain.com/u/HKWxj5ss

[Владимир001]

Match: T60 improvement
LC0-version: 0.26.3
LC0 options: Network Id: 65721 vs 63020
Options: Both: MinibatchSize=160
for 65721: MovesLeftMaxEffect=0.2, MovesLeftConstantFactor=0.0, MovesLeftThreshold = 0.0, MovesLeftSlope = 0.004, MovesLeftScaledFactor =1.0, MovesLeftQuadraticFactor = 0.0
for 63020: MovesLeftMaxEffect=0.0, MovesLeftConstantFactor=0.0, MovesLeftThreshold = 1.0, MovesLeftSlope = 0.000, MovesLeftScaledFactor =0.0, MovesLeftQuadraticFactor = 0.0
Hardware: 2threads, gtx1080
Time control: 800 nodes/move
Book: noob_3moves.epd
Tablebases: 6-man syzygy
Adjudication: Draw adjudication: move number: 50, move count: 5, score: 10 cp, Resign adjudication: Move count: 5, score: 1000cp, Tablebase adjudication
Software: Cutechess-gui
ordo: ./ordo-win64.exe -Q -N 0 -D -a 0 -A "lc0 t60 65721" -W -n4 -s500 -U "0,1,2,3,4,5,6,7,8,9,10" -j h2h.txt -C matrix.csv -p t60improvement.pgn
Comment: To see how much t60 improved since mlh is attached to training

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W     D    L  D(%)
   1 lc0 t60 65721       :       0   ----  1092.0    2000  54.6     100  393  1398  209  69.9
   2 lc0 63020 no_mlh    :     -32      8   908.0    2000  45.4     ---  209  1398  393  69.9

White advantage = 6.01 +/- 4.37
Draw rate (equal opponents) = 70.96 % +/- 1.01

https://pixeldrain.com/u/AYmRkfUq

[Владимир001]

Match Stockfish 13 Dev vs Lc0 JH J92-190
Tournament: Gauntlet
Binaries: BMI2 | Lc0-v0.26.1
Engine Options: Default
Hardware: i7 7700k 4CPU | 1080TI -- 1000:1 Ratio 20b
Time control: 60+1
Book: Balsa v2724
Tablebases: None
Adjudication: +-9 Win
Software: Cutechess-Cli
Sheets: https://docs.google.com/spreadsheets...#gid=201239930
Comments: Stockfish's performance against Lc0 in 1 month of progress

H2H
   # PLAYER                       :  RATING  ERROR  POINTS  PLAYED   (%)    W    D    L  D(%)  OppN  CFS(%)
   1 Stockfish 13 Dev [Oct 28]    :      56      9   162.0     280  57.9   49  226    5  80.7     1     100
   2 Leela Jhorthos J92-190       :       0      9   118.0     280  42.1    5  226   49  80.7     1     ---

White advantage = 25.71 +/- 8.70
Draw rate (equal opponents) = 88.82 % +/- 2.60


All
   # PLAYER                       :  RATING  ERROR  POINTS  PLAYED   (%)    W    D    L  D(%)  OppN  CFS(%)
   1 Stockfish 13 Dev [Oct 28]    :      56     17   162.0     280  57.9   49  226    5  80.7     1      81
   2 Stockfish 13 Dev [Sep 28]    :      44     16   157.5     280  56.3   43  229    8  81.8     1      96
   3 Stockfish 12                 :      19     18   105.5     200  52.8   26  159   15  79.5     1      96
   4 Leela Jhorthos J92-190       :       0      8   415.0     912  45.5   52  726  134  79.6     4      94
   5 Stockfish 11                 :     -19     20    72.0     152  47.4   16  112   24  73.7     1     ---

White advantage = 34.40 +/- 5.16
Draw rate (equal opponents) = 84.56 % +/- 1.40

[Владимир001]

Gauntlet: Lc0 26.3 dnnl + 703810 vs Stockfish 6,7,8 bmi2
Time control: 300s + 3s
Hardware: i5-8265u, 4CPU
Openings: Balsa_Top50.pgn
Stockfish: threads=4, hash=512
Lc0: threads=4, else default (no mlh)
Bench (startpos 10s): Lc0 4CPU = 980 nps, Stockfish 8 4CPU = 6 300 000 nps
Tbs and Adj.: syzygy 5-men, draw: 5 moves 5cp move 30 - resign: 5 moves 600cp
Comments: With this setup, Lc0 dnnl seems close to SF-7 strenght.
Games: https://gofile.io/d/iIlmVr

# PLAYER                       :  RATING  ERROR  PLAYED   (%)  CFS   W    D   L   D(%)
1 Stockfish 8   4CPU           :    69.9   42.4    100  61.00   99  32   58  10  58.00
2 Stockfish 7   4CPU           :     0.0   34.7    100  51.50   69  24   55  21  55.00
3 Lc0 26.3 dnnl 4CPU + 703810  :   -10.8   17.4    300  47.17   92  54  175  71  58.33
4 Stockfish 6   4CPU           :   -39.8   40.3    100  46.00  ---  15   62  23  62.00

White advantage = 61.36 +/- 14.16
Draw rate (equal opponents) = 62.07 % +/- 2.72

Engine               Depth   MIDG   EARLY    ENDG    LATE
Lc0 26.3 dnnl 4CPU    8.14   8.81 |  8.72 | 7.85  |  6.47
Stockfish 7   4CPU   29.38  25.34 | 25.13 | 28.37 | 41.87
Stockfish 6   4CPU   32.62  27.47 | 27.29 | 33.16 | 49.38
Stockfish 8   4CPU   31.86  27.43 | 27.06 | 30.66 | 50.83

[Владимир001]

NN-testrun of Lc0 0.26.3 J92-300 finished:

https://www.sp-cc.de/nn-vs-sf-testing.htm

[Владимир001]

Match: lc0.26.J92-300 vs Stockfish 12 4CPU - 100 rapid games
LC0-version: v26.0 cuda
LC0-options: --backend=multiplexing --cpuct=2.147 --cpuct-factor=2.815 --cpuct-base=18368 --fpu-value=0.443 --policy-softmax-temp=1.607 --moves-left-max-effect=0.1 --moves-left-threshold=0.5 --moves-left-slope=0.007 --moves-left-quadratic-factor=0 --moves-left-scaled-factor=1 --moves-left-constant-factor=0
Time control: 15min + 5s (CCRL 40/15)
Hardware: CPU i7-8700 4cores vs 1070Ti GPU
Book: Custom short lines played from both sides (!sheet4 for opening list)
Tablebase: 6 piece syzygy (DTZ + WDL) for both engines
Software: Arena
Speed: Leela ratio ~0.85; Lc0 npm~200K, SF npm~200M (based on 24x256 nets, SV nets much slower)
Context: Best score against SF12 so far, no net has ever won the match !sheet4 for test history

# PLAYER                   :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W    D    L
1 lc0.26.J92-300           :  3522.0   23.9    50.0     100    50      50    7   86    7
2 Stockfish_12_x64-bmi2    :  3522.0   ----    50.0     100    50     ---    7   86    7

lc0.26.J92-300 - Stockfish_12_x64-bmi2 : 50.0/100 7-7-86 (==1=======1==========0=0======1=1====0====================1=========10===0=0=====0==1===============)  50% -> 3522 ordo

Comparison to previous best:
lc0.26.J92-270 - Stockfish_12_x64-bmi2 : 49.5/100 3-4-93 (===0==========================1=====1================0===================0=0========1===============)  50% -> 3519 ordo

[Владимир001]

Развернуть для просмотра

Match: Match of of SF Dev 20092822 4CPU vs Lc0 J92-310
Lc0-version: v0.26.2 + v0.26.3 (since J92-260)
Lc0-options : -Backend=cudnn -NNCacheSize=2000000 -MaxOutOfOrderEvalsFactor=2.4 -MaxCollisionEvents=256 -TwoFoldDraws=true
MLH-settings S19 SuFi: -MovesLeftMaxEffect=0.2 -MovesLeftSlope=0.004 -MovesLeftThreshold=0.0 -MovesQuadraticFactor=0.0 -MovesScaledFactor=1.0
SF-version: Stockfish 20092822 x64 bmi2 (date: 28.09.2020)
SF/Cfish-options: threads=4 hash=2048 MB, contempt=24, NNUE=true, net=nn-baeb9ef2d183.nnue (default)
Time control: 5min + 5s
Hardware: notebook with mobile i7-7700HQ 4x2.8GHz (8 threads with HT on), mobile Geforce GTX 1050 TI 4GB
Book: Balsa_Special.pgn, 10 plies, sequentially, colors reversed
Tablebases: Syzygy 5-men on SSD
Speed: J92: 961 nps, SF Dev NNUE 4CPU: 3680k nps, Cfish NNUE 4CPU: 4198k nps
Adjudication: Draw at 50 moves rule, 3-fold repetition, after 200 moves, at 50 moves with eval = 0.00 ; 5-men Gaviota tablebase adjudication
Software: Arena 3.5.1, ponder = off
Elo rating: Ordo 1.2.6
Comments: J92-310 added. The table includes the ERET test results (111 positions, 60s time limit per position).

   # PLAYER                  :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W    D    L  D(%)  ERET test
   1 Cfish-NNUE 071020_4CPU  :     7.1   21.1    51.0     100  51.0      74    2   98    0  98.0  96/111
   2 Stockfish_20092822_4CPU :     0.0   ----   580.0    1000  58.0     100  172  816   12  81.6  95/111
   3 Lc0.net.JH.J92-280      :   -46.1   25.5    43.5     100  43.5      58    3   81   16  81.0  80/111
   4 Lc0.net.JH.J92-260      :   -49.7   25.3    43.0     100  43.0      58    1   84   15  84.0  81/111
   5 Lc0.net.JH.J92-270      :   -53.3   26.3    42.5     100  42.5      50    1   83   16  83.0  81/111
   6 Lc0.net.JH.J92-210      :   -53.3   26.1    42.5     100  42.5      76    1   83   16  83.0  81/111
   7 Lc0.net.JH.J92-220      :   -67.8   28.8    40.5     100  40.5      57    2   77   21  77.0  79/111
   8 Lc0.net.JH.J92-190      :   -71.5   29.7    40.0     100  40.0      50    1   78   21  78.0  79/111
   9 Lc0.net.JH.J92-310      :   -71.5   29.1    40.0     100  40.0      57    1   78   21  78.0  77/111     <= NEW
  10 Lc0.net.JH.J92-240      :   -75.2   29.3    39.5     100  39.5      75    0   79   21  79.0  80/111
  11 Stockfish_11_4CPU       :   -90.1   32.2    37.5     100  37.5     ---    0   75   25  75.0  91/111

https://pixeldrain.com/u/ps5V47TK

[свернуть]

[Владимир001]

Развернуть для просмотра

Match: ERET test (Eigenmann Rapid Engine Test) with 111 test positions
time control: analysis with time limit of 60s per move/position
LC0-version: v0.21.2 up to v0.26.3
LC0-options: -Backend=cudnn -NNcache=3000000 -MaxCcollisionEvents=256
MLH-settings S19 SuFi: -MovesLeftMaxEffect=0.2 -MovesLeftSlope=0.004 -MovesLeftThreshold=0.0 -MovesQuadraticFactor=0.0 -MovesScaledFactor=1.0
AB engine options: threads=4, hash=4096 MB
hardware: mobile Geforce GTX 1050 TI 4GB, mobile i7-7700HQ 4x2.8GHz (8 threads with HT)
software: Arena GUI, "Automatic Analysis" feature with EPD file.
Tablebases: Syzygy 5-men on SSD
Comment:https://glarean-magazin.ch/wp-conten...gine-Chess.zip
Results (solutions, percent):

 
A) CPU engines (mobile i7-7700HQ 4x2.8GHz):
Cfish 291020 NNUE 4CPU          :  97 = 87.4%   <== NEW, *net: nn-2eb2e0707c2b.nnue*
CorChess NNUE 1.3_311020 4CPU   :  97 = 87.4%   <== NEW, *net: nn-2eb2e0707c2b.nnue*
Stockfish 20102823 NNUE 4CPU    :  95 = 85.6%   <== NEW, ** TCEC Cup 7 **, *net: nn-2eb2e0707c2b.nnue*
Stockfish 20092822 NNUE 4CPU    :  95 = 85.6%   ** TCEC S19 Champion ** (net: nn-baeb9ef2d183.nnue)
Stockfish 12 4CPU (NNUE)        :  94 = 84.7%   *net: nn-82215d0fd0df.nnue*
BrainFish_200724_4CPU           :  92 = 82.9%   *last version before NNUE support is added*
Stockfish 20073110 4CPU         :  92 = 82.9%   *last version before NNUE support was added*
Stockfish 11 4CPU               :  91 = 82.9%
Stockfish 10 4CPU               :  90 = 81.1%
Komodo-13.2.5_4CPU              :  88 = 79.3%
Komodo 11.01 4CPU (freeware)    :  84 = 75.7%
Nemorino 6.00 NNUE 4CPU         :  77 = 69.4%   *net: nemorino_net_6.nnue*
SlowChess Classic 2.2 4CPU      :  77 = 69.4%
Xiphos 0.6 4CPU                 :  76 = 68.4%
Ethereal 12.75 pext-avx2 4CPU   :  73 = 65.8%
Igel 2.8.0 NNUE 4CPU            :  72 = 64.9%   *default net (night nurse v0.2.5b)*
Laser 1.7 4CPU                  :  72 = 64.9%
Komodo-MCTS-13.2.5_4CPU         :  62 = 55.9%
Lc0.net.703350-blas_4CPU        :  60 = 54.1%
Lc0.net.591003-blas_kiudee_4CPU :  60 = 54.1%
Fire 7.1 4CPU                   :  60 = 54.1%
Lc0.net.11258-112x9se_blas_4CPU :  59 = 53.2%
Lc0.net.DK.Bad_Gyal_7_blas_4CPU :  57 = 51.4%
Lc0.net.LD2_blas_4CPU  (10x128) :  56 = 50.4%

[свернуть]

[Alvir]

Очередная сеть от JHORTHOS J92-320 :
https://storage.lczero.org/files/net...ontrib/J92-320

[Владимир001]

Развернуть для просмотра

Match Gauntlet
LC0-version: lc0-v0.26.3 cudnn-version
LC0 options: Standard option except Threads=1, NNCacheSize=0, MinibatchSize=1.
Time control: 1 Node/move
Hardware: i7-4790 with 16 GByte RAM + GTX1650
Speed: Not important for this type of testing.
Book: Chad-opening book with 100 000 different openings with the most popular human openings.
Book options: Opening order: Sequential. Play each opening twice. Play all opening in the book both for White and Black.
Tablebases: No Tablebases.
Adjudication: Only 5-men syzygy
Software: CuteChess 1.2.0
https://github.com/cutechess/cutechess/releases
Graphic tools is programmed by glbchess64, in Python programming tools.
Comment: This is result out of Ordo 1.2.6 https://github.com/michiguel/Ordo/releases/tag/v1.2.6
Ordo string: ordo.exe -Q -N 0 -D -a 0 -A "J92-145" -W -n4 -s500 -G -V -U "0,1,2,3,4,5,6,7,8,9,10" -j stats.txt -o rating.txt -p games.pgn
Results: pgn-file: https://ufile.io/vf2ad3u1
Explanation: Trying to find the best net in clean policy Elo.
```

# PLAYER     :  RATING  ERROR    POINTS  PLAYED   (%)  CFS(%)      W      D      L  D(%)
1 J92-310    :      19      1  105512.0  200000  52.8     100  68186  74652  57162  37.3
2 J92-145    :       0      1   94488.0  200000  47.2     ---  57162  74652  68186  37.3

White advantage = 29.57 +/- 0.57
Draw rate (equal opponents) = 37.63 % +/- 0.10

My list until now (Elo against J92-145) 1 Node/move:

White advantage = 23.98 +/- 0.57
Draw rate (equal opponents) = 36.51 % +/- 0.10
J92-310:             19     Elo    (Test with 200k games)
J92-270:             17     Elo    (Test with 200k games)
J92-280:             11     Elo    (Test with 200k games)
J92-260:             11     Elo    (Test with 200k games)
J92-290:             10     Elo    (Test with 200k games)
J92-240:              8     Elo    (Test with 200k games)
J92-210:              5     Elo    (Test with 200k games)
J92-220:              3     Elo    (Test with 200k games)
J92-205:              3     Elo    (Test with 200k games)
J92-230:              2     Elo    (Test with 200k games)
J92-190:              1     Elo    (Test with 200k games)
J92-145:              0     Elo    (Reference)
J92-100:             -2     Elo    (Test with 200k games)

[свернуть]

[Владимир001]

Развернуть для просмотра

Match: SmartPruningFactor
LC0 version: 0.26.3
LC0 options: threads=1, minibatch-size=192
MLH options: moves-left-max-effect=0.2, moves-left-threshold=0, moves-left-slope=0.004, moves-left-scaled-factor=1, moves-left-quadratic-factor=0
Time control: 1s+0.1s
Net: 703810
Book: custom-3k (1-3 ply)
Hardware: gtx970 + i5-4570
Tablebases: syzygy 5-man
Adjudication: -draw movenumber=50 movecount=5 score=10 -resign movecount=5 score=700 twosided=true
Software: cutechess-cli
Comment: default smart pruning value (1.33) doesnt seem optimal

   # PLAYER                   :  RATING  ERROR  POINTS  PLAYED   (%)  CFS(%)    W     D    L  D(%)
   1 lc0-smartpruning_1.00    :     7.7    8.5  1022.0    2000    51      68  340  1364  296    68
   2 lc0-smartpruning_0.75    :     4.7    8.8  1013.5    2000    51      86  348  1331  321    67
   3 lc0-smartpruning_1.33    :     0.0   ----  1964.5    4000    49     ---  617  2695  688    67

White advantage = -5.00 +/- 3.17
Draw rate (equal opponents) = 67.43 % +/- 0.73

https://pixeldrain.com/u/CswZff6Q

[свернуть]

[Владимир001]

NN-testrun of Lc0 0.26.3 65732 finished:

https://www.sp-cc.de/nn-vs-sf-testing.htm

[Владимир001]

Развернуть для просмотра

Match: J92-320 vs stockfish 12 in fixed nodes
LC0 version: 0.26.2, built with cuda11 + cudnn7
LC0 options: cudnn-fp16, 1 thread, default for the rest
SF options: 1 thread, 1GB Hash.
Time control: Leela: 1 kn/move, SF 12: 400Kn/move
Hardware: RTX 2070, i7-7700
Book: Balsa 2724 openings
Tablebases: 6-man TB for both.
Adjudication: 6-man TB, -draw movenumber=50 movecount=5 score=8 -resign movecount=5 score=1000
Software:cutechess-cli
Comments: J92-320 got -6 elo vs SF 12, unimpressive here.

Leela: 1 thread, 1Kn/move
SF 12: 1 thread, 400Kn/move

Balsa 2724

   # PLAYER                :  RATING  ERROR   POINTS  PLAYED   (%)  CFS(%)      W      D      L
-  1 Stockfish_12.400k     :       0   ----  30408.5   59928  50.7      57  11905  37007  11016
   2 lc0.net.J92-270.1k    :      -1      6   2720.0    5448  49.9      68   1063   3314   1071
   3 lc0.net.J92-290.1k    :      -2      6   2705.5    5448  49.7      53   1012   3387   1049
   4 lc0.net.J92-260.1k    :      -3      6   2703.0    5448  49.6      54   1004   3398   1046
   5 lc0.net.J92-280.1k    :      -3      6   2699.5    5448  49.6      55   1045   3309   1094
   6 lc0.net.J92-190.1k    :      -4      6   2696.0    5448  49.5      54    996   3400   1052
   7 lc0.net.J92-250.1k    :      -4      6   2693.0    5448  49.4      61   1015   3356   1077
   8 lc0.net.J92-230.1k    :      -5      6   2684.0    5448  49.3      51    980   3408   1060
   9 lc0.net.J92-300.1k    :      -5      6   2683.5    5448  49.3      55   1009   3349   1090
+ 10 lc0.net.J92-320.1k    :      -6      6   2679.5    5448  49.2      75   1004   3351   1093
  11 lc0.net.J92-220.1k    :      -9      6   2656.0    5448  48.8      96    978   3356   1114
  12 lc0.net.J92-100.1k    :     -16      6   2599.5    5448  47.7     ---    910   3379   1159

https://pixeldrain.com/u/uk4RWbBn

[свернуть]

[Владимир001]

Развернуть для просмотра

Match: J92 vs SF 12
Nodes: Lc0 26.3 cudnn-fp16 = 5000, Stockfish 12 modern = 2 500 000
Openings: TCEC SuFi 10-19 (500 pos.)
Hardware: Ryzen 7 3750H, GTX 1660 Ti (J92-270: Ryzen 9 4900H, RTX 2060)
Lc0: nncache=100000, threads=1
MLH: moves-left-max-effect=0.2, moves-left-threshold=0, moves-left-slope=0.004, moves-left-constant-factor=0, moves-left-scaled-factor=1, moves-left-quadratic-factor=0
SF 12: threads=2, hash=64
TBs and adj.: syzygy 5-men, draw 5 moves 5cp move 30, resign (two-sided) 5 moves 600cp
Games: https://gofile.io/d/hL0fjz

# PLAYER                : RATING  ERROR  PLAYED    (%)  CFS   W     D    L   D(%)
1 lc0-263-j92-310-5k    :   13.5   13.7   1000   51.80   69  247   542  211  54.20
2 lc0-263-j92-270-5k    :    9.4   12.9   1000   51.25   72  230   565  205  56.50
3 lc0-263-j92-280-5k    :    3.7   13.5   1000   50.50   55  239   532  229  53.20
4 lc0-263-J92-190-5k    :    2.6   14.2   1000   50.35   64  227   553  220  55.30
5 stockfish-12-2T-2500k :    0.0   ----   4000   49.02  ---  865  2192  943  54.80

White advantage = 90.32 +/- 3.69
Draw rate (equal opponents) = 59.40 % +/- 0.75

Engine                 Depth  MIDG  EARLY   ENDG   LATE
lc0-263-j92-310-5k     7.99   8.67 | 8.74 | 7.25 | 6.27
lc0-263-J92-190-5k     8.01   8.61 | 8.71 | 7.36 | 6.39
lc0-263-j92-270-5k     8.06   8.72 | 8.94 | 7.34 | 6.48
lc0-263-j92-280-5k     8.06   8.73 | 8.70 | 7.35 | 6.55
stockfish-12-2T-2500  25.50  22.51 | 22.67| 26.62| 38.37

[свернуть]

[Владимир001]

Развернуть для просмотра

Match: J92 vs SF-12 6cpu
Time control: 10s + 1s, Ponder=ON
Openings: SuperGM_4mvs_500.pgn (500 pos.)
Hardware: Ryzen 9 4900H, RTX 2060
Lc0 26.3 cuda: nncache=1000000, threads=2, minibatch-size=112, max-prefetch=50, max-collision-events=50
MLH: moves-left-max-effect=0.2, moves-left-threshold=0, moves-left-slope=0.004, moves-left-constant-factor=0, moves-left-scaled-factor=1.0, moves-left-quadratic-factor=0
SF 12: threads=6, hash=256
TBs and adj.: syzygy 5-men, draw 5 moves 5cp move 30, resign (two-sided) 5 moves 600cp
Comments: J92-310 on top again. Higher draw rate (see previos post) leads to smaller reported errors by ORDO.
Games: https://gofile.io/d/B7ukcF

# PLAYER                  :  RATING  ERROR  PLAYED  (%)   CFS    W    D    L   D(%)
1 stockfish-12-6cpu       :     0.0   ----   5000  52.63  100  514  4235  251  84.70
2 lc0-263-j92-310         :   -10.9    8.0   1000  48.45   57   53   863   84  86.30
3 lc0-263-j92-190         :   -11.9    7.7   1000  48.30   61   61   844   95  84.40
4 lc0-263-j92-270         :   -13.3    6.4   1000  48.10   73   53   856   91  85.60
5 lc0-263-j92-280         :   -16.5    8.3   1000  47.65  100   49   855   96  85.50
6 lc0-263-svjio-3972_20k  :   -39.8    8.5   1000  44.35  ---   35   817  148  81.70

White advantage = 10.35 +/- 1.87
Draw rate (equal opponents) = 85.98 % +/- 0.53

Engine                Depth  MIDG   EARLY   ENDG   LATE
lc0-263-j92-190       8.66   9.50 |  9.83 | 7.94 | 6.73
lc0-263-j92-280       8.72   9.58 | 10.16 | 8.11 | 6.56
lc0-263-j92-310       8.76   9.67 |  9.88 | 7.95 | 6.62
lc0-263-j92-270       8.77   9.64 |  9.92 | 8.09 | 6.71
lc0-263-svjio-3972    9.09   9.75 |  9.97 | 8.69 | 7.11
stockfish-12-6cpu    31.71  26.26 | 26.90 | 33.56| 48.80

[свернуть]

[Омегазеро]

А вот и свежая сеть J92-330 https://storage.lczero.org/files/net...ontrib/J92-330