Jjosh

Leela developer/tester/enthusiast. Gauging interest on current projects. Working on supervised learning now with "Leelenstein", combined with some other lc0 branch things. Leelenstein started out mostly CCRL games, with some 40b and T30 games. Currently uses CCRL games + more T30 games + SGDR + GGT + SE, but started with the same weights as the previous to retain some information from those other games. (There have been several different SL runs with some nice nets picked from each, and some slightly different datasets.) Trade Penalty is also recommended (and used at CCCC in current tournament). SGDR is new method for learning rate scheduling. <a href="https://arxiv.org/abs/1608.03983">https://arxiv.org/abs/1608.03983</a> GGT is new SL optimizer. <a href="https://arxiv.org/pdf/1806.02958.pdf">https://arxiv.org/pdf/1806.02958.pdf </a>Gradient Noise (adding to improve start and generalization) <a href="https://arxiv.org/abs/1511.06807">https://arxiv.org/abs/1511.06807</a> Initialization <a href="https://medium.com/@prateekvishnu/xavier-and-he-normal-he-et-al-initialization-8e3d7a087528">https://medium.com/@prateekvishnu/xavier-and-he-normal-he-et-al-initialization-8e3d7a087528</a> Tested significantly stronger than Gull 3 on 30CPU with a 2080 TI, and beat the Sufi 32930 in a tactics test, but lost head to head. EDIT: 8.0 added batch renormalization <a href="https://arxiv.org/abs/1702.03275">https://arxiv.org/abs/1702.03275</a> 9.0 added a new optimizer 10 added label smoothing for policy 11.0 added some t40 data which didn't seem to help, but 11.1 changed some smoothing parameters and gained a fair bit 12.0 started using Leelenstein to fix the training data. Mostly value head tweaks. Changed policy head. 13.0 added some t40 games, changed value head, starting to do more policy data changes. Has gone from being weaker than 32930 H2H to stronger than 41800.

Free

Pledge/Show support for Leelenstein. Get early access to these networks

Leelenstein Fan

Show extra support. Possible exclusive posts, and access to network checkpoints at some LR cycle points. (When LR drops to zero). Preview of future performance that is possibly (but unlikely) stronger than final.

Choose your membership