Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.6 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 1516 2426878 2426658 2026-06-15T12:17:57Z Stashbot 7414 Barras: -ChanServ/#cvn-wp-en- Barras set flags +AV on slakr 2426878 wikitext text/x-wiki saam6ue6wmhka6uwep2hb6mj18buj9c 2426886 2426878 2026-06-15T12:28:19Z Stashbot 7414 Barras: -ChanServ- Flags +AV were set on TenshiHinanawi in #cvn-sw. 2426886 wikitext text/x-wiki 6tz1aflfcz2erxe2jfew61fmnttln8g 2426887 2426886 2026-06-15T12:28:27Z Stashbot 7414 Barras: -ChanServ- Flags +AV were set on TenshiHinanawi in #cvn-ja. 2426887 wikitext text/x-wiki mmso3kcm594pvoozk69ljj8nc1zdfl1 2426889 2426887 2026-06-15T12:31:15Z Stashbot 7414 Barras: <Barras> CVNBot1 restart 2426889 wikitext text/x-wiki afjjob2vtdsuvu2qrkbe8hc74990q9r 498 3086 2426862 2426861 2026-06-15T11:59:11Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-gateway-3 from 1.31.14 to 1.32.13 (T427919) 2426862 wikitext text/x-wiki sh719ayb0ida0afl16muvgdoi9o3wt4 2426863 2426862 2026-06-15T12:01:32Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade_bastions for tools-bastion-15.tools.eqiad1.wikimedia.cloud, tools-bastion-14.tools.eqiad1.wikimedia.cloud (T427919) 2426863 wikitext text/x-wiki 96s0ih5zomhueb06k9sn38v0n487ui4 2426864 2426863 2026-06-15T12:02:10Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_bastions (exit_code=0) for tools-bastion-15.tools.eqiad1.wikimedia.cloud, tools-bastion-14.tools.eqiad1.wikimedia.cloud (T427919) 2426864 wikitext text/x-wiki kaehbj37z5t3my5ndsed7gdflsg9yil 2426869 2426864 2026-06-15T12:06:57Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=0) for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-wor 2426869 wikitext text/x-wiki inyx9is7wqu1q5j0ph2g37498yvgjo4 2426877 2426869 2026-06-15T12:17:04Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=0) for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-wor 2426877 wikitext text/x-wiki e5qj4c3dlb4nlijc5qpq95fvfm80ucf 2426879 2426877 2026-06-15T12:17:58Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade_workers (exit_code=0) for tools-k8s-worker-nfs-1, tools-k8s-worker-nfs-10, tools-k8s-worker-nfs-11, tools-k8s-worker-nfs-12, tools-k8s-worker-nfs-13, tools-k8s-worker-nfs-14, tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-17, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-2, tools-k8s-worker-nfs-21, tools-k8s-worker-nfs-22, tools-k8s-wor 2426879 wikitext text/x-wiki 02qw2w7j97p953b56n2ek0mrli7ofjl 2426880 2426879 2026-06-15T12:18:10Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.run_tests 2426880 wikitext text/x-wiki kgoefhgi05ozjtjtzi27ovklwzgzorj 2426899 2426880 2026-06-15T12:41:26Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99) 2426899 wikitext text/x-wiki n6m6wfahnftkjjiwgcfqeab7dj06tqy 2427085 2426899 2026-06-15T17:13:20Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld 2427085 wikitext text/x-wiki 1acjl7hqwk9dlpqyb7i27kjy98l2h21 2427088 2427085 2026-06-15T17:26:58Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld 2427088 wikitext text/x-wiki 55mtcagromi47ux1ptn98fyws3xyg22 0 3146 2427292 141765 2026-06-16T09:44:07Z Effie Mouzeli (WMF) 12880 Removed redirect to [[Application servers]] 2427292 wikitext text/x-wiki m6eflce4cmea27ytin8oig9xyu9g2m8 0 3240 2427005 2398863 2026-06-15T14:20:57Z TerrellTan 56562 test 2427005 wikitext text/x-wiki drhe5h9qdx2rjf0aru1rd4obnvj26ln 2427006 2427005 2026-06-15T14:21:39Z TerrellTan 56562 2427006 wikitext text/x-wiki nv4b0b1bj5zrck1ofap981d5w108p06 0 4108 2426873 2426787 2026-06-15T12:11:17Z ScheduleDeploymentBot 37566 Add [[gerrit:1301401]] to Monday, June 15 UTC afternoon backport window 2426873 wikitext text/x-wiki 5tszfltuoc5920huh0kvgppvgpbj0p8 2426874 2426873 2026-06-15T12:11:37Z ScheduleDeploymentBot 37566 Add [[gerrit:1301451]] to Monday, June 15 UTC afternoon backport window 2426874 wikitext text/x-wiki 2rsugq9ufvmjb6xj0xtoxh7vxtghwtk 2426905 2426874 2026-06-15T12:55:38Z ScheduleDeploymentBot 37566 Add [[gerrit:1300858]] to Monday, June 15 UTC afternoon backport window 2426905 wikitext text/x-wiki c7bcr94wa1lixwnoin72ylrmfbif6oi 2426906 2426905 2026-06-15T12:55:52Z ScheduleDeploymentBot 37566 Add [[gerrit:1300872]] to Monday, June 15 UTC afternoon backport window 2426906 wikitext text/x-wiki 181ry7go2spfvqpct9rmyrfv4in9krg 2426907 2426906 2026-06-15T12:55:59Z ScheduleDeploymentBot 37566 Add [[gerrit:1301389]] to Monday, June 15 UTC afternoon backport window 2426907 wikitext text/x-wiki 108hxrtd8lh1zbhnp8qe9s4cptwxgdw 2426928 2426907 2026-06-15T13:19:40Z Jdforrester (WMF) 468 /* {{Deployment_day|date=2026-06-15}} */ Five done. 2426928 wikitext text/x-wiki iim79nt1qp45oo6m9i28b0q07f78inp 2426949 2426928 2026-06-15T13:39:34Z Jdforrester (WMF) 468 /* {{Deployment_day|date=2026-06-15}} */ Remaining ones done. 2426949 wikitext text/x-wiki 2lrh4ve61058xhsfs0r6ys0tmi57bq4 2426967 2426949 2026-06-15T13:55:14Z ScheduleDeploymentBot 37566 Add [[gerrit:1300245]] to Monday, June 15 UTC late backport window 2426967 wikitext text/x-wiki fm0fs3md0pjsnf8adugt4ictd0hz7th 2427076 2426967 2026-06-15T16:46:06Z ScheduleDeploymentBot 37566 Add [[gerrit:1298875]] to Tuesday, June 16 UTC afternoon backport window 2427076 wikitext text/x-wiki ehq6pwp6gzr0eseso4uaj40g1lmms32 2427115 2427076 2026-06-15T18:45:33Z ScheduleDeploymentBot 37566 Add [[gerrit:1302232]] to Monday, June 15 UTC late backport window 2427115 wikitext text/x-wiki p72cxlev9gnny73nx5wu61r2fy3g0nm 2427203 2427115 2026-06-16T06:03:11Z Christoph Jauera (WMDE) 3635 Adding 1302170 2427203 wikitext text/x-wiki n70k8b61bsm1b8jmyj2ikaz7b7nxk02 2427204 2427203 2026-06-16T06:03:48Z ScheduleDeploymentBot 37566 Add [[gerrit:1302629]] to Tuesday, June 16 UTC morning backport window 2427204 wikitext text/x-wiki 2qd2iziz33adsuo41r3263gwxizqavm 2427237 2427204 2026-06-16T08:46:19Z ScheduleDeploymentBot 37566 Add [[gerrit:1302739]] to Wednesday, June 17 UTC morning backport window 2427237 wikitext text/x-wiki 8etkapn1uuzdgscxfxpkpl7pka8087r 2427238 2427237 2026-06-16T08:46:37Z ScheduleDeploymentBot 37566 Add [[gerrit:1302739]] to Wednesday, June 17 UTC morning backport window 2427238 wikitext text/x-wiki g0eks7lw523hg20i3sabpb91436qubw 2427240 2427238 2026-06-16T08:47:36Z ScheduleDeploymentBot 37566 Add [[gerrit:1302743]] to Wednesday, June 17 UTC morning backport window 2427240 wikitext text/x-wiki qn1pp4rcd4bm2xqdzq5d333nfeoozhv 2427243 2427240 2026-06-16T08:49:17Z APatro (WMF) 38793 /* {{Deployment_day|date=2026-06-17}} */ 2427243 wikitext text/x-wiki 9w975ort0py3rtykhyo1rkpzqzmldus 2427282 2427243 2026-06-16T09:35:09Z ScheduleDeploymentBot 37566 Add [[gerrit:1299626]] to Tuesday, June 16 UTC afternoon backport window 2427282 wikitext text/x-wiki 4ryp35oib4biv6qojwom6br9c449dl4 2427299 2427282 2026-06-16T10:06:50Z ScheduleDeploymentBot 37566 Add [[gerrit:1302197]] to Tuesday, June 16 UTC afternoon backport window 2427299 wikitext text/x-wiki 3ndfi5ugyh6qheu0f3etvf78k7bbyqv 2427332 2427299 2026-06-16T10:52:32Z ScheduleDeploymentBot 37566 Add [[gerrit:1298293]] to Wednesday, June 17 UTC afternoon backport window 2427332 wikitext text/x-wiki 5ccp3acwzohthewnpxs5408r1d9ywkk 2427333 2427332 2026-06-16T10:53:01Z ScheduleDeploymentBot 37566 Add [[gerrit:1299943]] to Wednesday, June 17 UTC afternoon backport window 2427333 wikitext text/x-wiki cptiscaosqvg7cggtzvchrh6x39rmh8 2427337 2427333 2026-06-16T11:36:42Z ScheduleDeploymentBot 37566 Add [[gerrit:1302792]] to Tuesday, June 16 UTC afternoon backport window 2427337 wikitext text/x-wiki bg36loq0zswc87by7ea0hpd0b6464uf 2427338 2427337 2026-06-16T11:36:52Z ScheduleDeploymentBot 37566 Add [[gerrit:1302793]] to Tuesday, June 16 UTC afternoon backport window 2427338 wikitext text/x-wiki fiogwoc89186vc0q0i5okm8sz8p21el 0 4874 2427117 2424027 2026-06-15T18:58:42Z Matma Rex 562 /* Schedule */ Bigger diagram 2427117 wikitext text/x-wiki rfvjm8yg0v2k5e8qzzmgox3e6w9icdh 498 6726 2427077 2426221 2026-06-15T16:58:07Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld 2427077 wikitext text/x-wiki 01win5i50e2fpf5n7akdhux9o581k9w 2427083 2427077 2026-06-15T17:09:53Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld 2427083 wikitext text/x-wiki 2gucuyfbrhpwszscnhkjwsv4yk3z17v 0 7919 2426865 2426859 2026-06-15T12:02:58Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1251.eqiad.wmnet with reason: host reimage 2426865 wikitext text/x-wiki 7wuk15o47mct3n7p5worjishwz0wncq 2426866 2426865 2026-06-15T12:05:42Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host kafka-logging1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426866 wikitext text/x-wiki seemeu1lo5eyh7i3x2ouydkynur0fwc 2426867 2426866 2026-06-15T12:06:09Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host kafka-logging1007.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426867 wikitext text/x-wiki d6hxk40qdu4o8w5d52cv1m237mszaaq 2426868 2426867 2026-06-15T12:06:37Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host kafka-logging1007.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426868 wikitext text/x-wiki nrngekigqdf01q131cwo7mez3xk4ksc 2426870 2426868 2026-06-15T12:06:59Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] START helmfile.d/services/proton: apply 2426870 wikitext text/x-wiki bd88ucm4n4sn4isfqoap9mbohi1kgxu 2426871 2426870 2026-06-15T12:10:00Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1251.eqiad.wmnet with reason: host reimage 2426871 wikitext text/x-wiki 8kvlz0sd3l7prbortf4pjt8ot4gffao 2426872 2426871 2026-06-15T12:10:52Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] DONE helmfile.d/services/proton: apply 2426872 wikitext text/x-wiki 0qu4jwd41ppqqyussywgyp6yrkqhm6y 2426875 2426872 2026-06-15T12:12:42Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2216.codfw.wmnet with reason: host reimage 2426875 wikitext text/x-wiki pnnlucckcup9uc95uqb4tc78fzxdnr8 2426876 2426875 2026-06-15T12:15:35Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host kafka-logging1007.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426876 wikitext text/x-wiki 50b4z7ytgprmqw15zk3ni0lnrzsgunm 2426881 2426876 2026-06-15T12:18:47Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2216.codfw.wmnet with reason: host reimage 2426881 wikitext text/x-wiki 0goe48hsdg862zju5y17a99pw9ztt6p 2426882 2426881 2026-06-15T12:21:47Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] START helmfile.d/services/proton: apply 2426882 wikitext text/x-wiki ixganodr877t73dcik0t4dnqsp8kaz6 2426883 2426882 2026-06-15T12:23:52Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] DONE helmfile.d/services/proton: apply 2426883 wikitext text/x-wiki 6w5owlgk5zfas337eqkjq5bdd12iwaf 2426884 2426883 2026-06-15T12:26:53Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1251.eqiad.wmnet with OS trixie 2426884 wikitext text/x-wiki j1j25rna1wlxds3woxpaicmcbq5193k 2426885 2426884 2026-06-15T12:27:30Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host kafka-logging1007.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426885 wikitext text/x-wiki 2efv75g9f88gp5qnr0thoedbl4rt3ep 2426888 2426885 2026-06-15T12:31:01Z Stashbot 7414 mszwarc@deploy1003: Started scap sync-world: Backport for [[gerrit:1302124|Extract a service that initiates SI signal matching (T428557)]], [[gerrit:1302125|Trigger Suggested Investigations when client hints are saved (T428557)]] 2426888 wikitext text/x-wiki igtd37engf8pxmqk6q5eptqcujs8p5e 2426890 2426888 2026-06-15T12:32:49Z Stashbot 7414 mszwarc@deploy1003: mszwarc: Backport for [[gerrit:1302124|Extract a service that initiates SI signal matching (T428557)]], [[gerrit:1302125|Trigger Suggested Investigations when client hints are saved (T428557)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2426890 wikitext text/x-wiki b53gzguyrx65vftfn52rfeym9396f3e 2426891 2426890 2026-06-15T12:32:58Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host kafka-logging1008.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426891 wikitext text/x-wiki dg5ipiz8ly9po8ranp3ifvcdslxcedk 2426892 2426891 2026-06-15T12:34:23Z Stashbot 7414 mszwarc@deploy1003: mszwarc: Continuing with deployment 2426892 wikitext text/x-wiki r0yhq3invx2nh08zmw7pn10t9elmu54 2426893 2426892 2026-06-15T12:34:52Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426893 wikitext text/x-wiki iawqt05t8qzzeqt5e4wkk875bn4ixs0 2426894 2426893 2026-06-15T12:35:52Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2216.codfw.wmnet with OS trixie 2426894 wikitext text/x-wiki fqw05efuq2tyg3t4l6llt05ccmtqoyr 2426895 2426894 2026-06-15T12:37:54Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1251: Migration of db1251.eqiad.wmnet completed 2426895 wikitext text/x-wiki 7uxndlq1t6sxxlyr534jeh3asqxomzq 2426896 2426895 2026-06-15T12:38:43Z Stashbot 7414 mszwarc@deploy1003: Finished scap sync-world: Backport for [[gerrit:1302124|Extract a service that initiates SI signal matching (T428557)]], [[gerrit:1302125|Trigger Suggested Investigations when client hints are saved (T428557)]] (duration: 07m 42s) 2426896 wikitext text/x-wiki fch6r53ztdeo04ojwvgavok3p28z7uw 2426897 2426896 2026-06-15T12:40:07Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es2036: Migration of es2036.codfw.wmnet completed 2426897 wikitext text/x-wiki it9habstyrl3l5mdb39q8gb7ceepyri 2426898 2426897 2026-06-15T12:40:09Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2426898 wikitext text/x-wiki 0mk4p4074jjwrlf16g5o1v63am0mfbv 2426900 2426898 2026-06-15T12:43:48Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host kafka-logging1008.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426900 wikitext text/x-wiki 8l4iq9de3heih8ddx06fifhddpcg3ft 2426901 2426900 2026-06-15T12:45:37Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1078.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426901 wikitext text/x-wiki 9e2jppabq4w1e367t5t6gqyct4inw0b 2426902 2426901 2026-06-15T12:46:37Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db2216: Migration of db2216.codfw.wmnet completed 2426902 wikitext text/x-wiki so3nf4wtv0zpx1iqzu527aqburmpgmw 2426904 2426902 2026-06-15T12:48:40Z Stashbot 7414 moritzm: installing augeas security updates 2426904 wikitext text/x-wiki btaroy3mgk4s1c1x63muu074ik71s2d 2426908 2426904 2026-06-15T12:57:54Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2426908 wikitext text/x-wiki cihj0qmbh063z48oi08b732kxu6wcdt 2426909 2426908 2026-06-15T12:57:57Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2426909 wikitext text/x-wiki 7rscilyvsnf9xlz5hzcq51rar6vmtub 2426910 2426909 2026-06-15T13:02:18Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2426910 wikitext text/x-wiki 37ux7ageo9q82hvf1bow730ozd6h4s3 2426911 2426910 2026-06-15T13:05:01Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426911 wikitext text/x-wiki jc5247pkdqygu57q18vxrxpd0wf8yfi 2426912 2426911 2026-06-15T13:05:16Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2426912 wikitext text/x-wiki 9jtjzlbj3fozkv9qdyh9mwg99dbzoas 2426913 2426912 2026-06-15T13:08:18Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2426913 wikitext text/x-wiki m8woi8uf56ruky3teopx869j8ulxd5l 2426915 2426913 2026-06-15T13:08:43Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2426915 wikitext text/x-wiki ft1lxdlsntx1wv38js7osc7yu99tyo9 2426916 2426915 2026-06-15T13:10:34Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2426916 wikitext text/x-wiki o31b931qhjv08nc3jtyzsw6bnplkeln 2426917 2426916 2026-06-15T13:10:59Z Stashbot 7414 moritzm: installing Linux 6.1.174 on Bookworm hosts 2426917 wikitext text/x-wiki fm0kq9tuecpurw06rcov6q02c0i8lna 2426918 2426917 2026-06-15T13:12:07Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1293173|Configure wgOAuthAutoApprove['protocols'] (T412542 T426614)]], [[gerrit:1300873|jawiki: remove four rights from the eliminator group (T428942)]], [[gerrit:1301401|Deploy PRV to 6 wikis (T429038)]], [[gerrit:1300858|[abstractwiki] Set wgForceUIMsgAsContentMsg for sidebar messages (T427730)]], [[gerrit:1300872|abstractwiki: Temporary config fo 2426918 wikitext text/x-wiki pyhr43yfcr9tzgholgkya2wirba1at2 2426919 2426918 2026-06-15T13:12:14Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426919 wikitext text/x-wiki 75c6tft7m6qkb3ap49vwlbnmbqkbnsq 2426920 2426919 2026-06-15T13:13:15Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426920 wikitext text/x-wiki h5e58tmv0jzu8n6u4e4isqoo4evh6qw 2426921 2426920 2026-06-15T13:13:52Z Stashbot 7414 jforrester@deploy1003: arlolra, matmarex, jforrester, dragoniez: Backport for [[gerrit:1293173|Configure wgOAuthAutoApprove['protocols'] (T412542 T426614)]], [[gerrit:1300873|jawiki: remove four rights from the eliminator group (T428942)]], [[gerrit:1301401|Deploy PRV to 6 wikis (T429038)]], [[gerrit:1300858|[abstractwiki] Set wgForceUIMsgAsContentMsg for sidebar messages (T427730)]], [[gerrit:1300872|abstractwiki: Te 2426921 wikitext text/x-wiki fctxz8whxf0ipkbr1l7d276gs34899m 2426923 2426921 2026-06-15T13:17:51Z Stashbot 7414 jforrester@deploy1003: arlolra, matmarex, jforrester, dragoniez: Continuing with deployment 2426923 wikitext text/x-wiki ixu31x2s0labw32zx3jduxkj86sjqel 2426924 2426923 2026-06-15T13:18:08Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1079.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426924 wikitext text/x-wiki g8nocnpwviw58ubu7zfsx9jjvwzkogt 2426927 2426924 2026-06-15T13:18:57Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426927 wikitext text/x-wiki my9repw2atmmxrj94hgfikezc6oy0i4 2426929 2426927 2026-06-15T13:20:38Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2426929 wikitext text/x-wiki 2uy2zahwym3b28g24ye4p6cl4hifjrd 2426931 2426929 2026-06-15T13:22:15Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1293173|Configure wgOAuthAutoApprove['protocols'] (T412542 T426614)]], [[gerrit:1300873|jawiki: remove four rights from the eliminator group (T428942)]], [[gerrit:1301401|Deploy PRV to 6 wikis (T429038)]], [[gerrit:1300858|[abstractwiki] Set wgForceUIMsgAsContentMsg for sidebar messages (T427730)]], [[gerrit:1300872|abstractwiki: Temporary config f 2426931 wikitext text/x-wiki 8a80iojgdkxzmtnj1tzlrlhl8lfrf4w 2426932 2426931 2026-06-15T13:23:24Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1251: Migration of db1251.eqiad.wmnet completed 2426932 wikitext text/x-wiki 70zo3nho4nknbxg9a5u27amcexudq4b 2426933 2426932 2026-06-15T13:23:26Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2426933 wikitext text/x-wiki e4mwqh9u9ltalubohmd4ugqc5i5v5au 2426934 2426933 2026-06-15T13:24:59Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "Haproxy provenance maps in HP; UX changes - oblivian@cumin1003" 2426934 wikitext text/x-wiki ruky3tdbx64sxq46ewrl62mrekxv5yq 2426935 2426934 2026-06-15T13:25:01Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: Haproxy provenance maps in HP; UX changes - oblivian@cumin1003 2426935 wikitext text/x-wiki 0c1dcwvr87lguqmqrvmbhizbu1487at 2426936 2426935 2026-06-15T13:25:17Z Stashbot 7414 topranks: cr2-esams, reconfigure chassis fpc to set port 0 to 100G T427056 2426936 wikitext text/x-wiki n9ub1onuk191yotd5jsqtmfcp1zcp5v 2426937 2426936 2026-06-15T13:25:59Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: Haproxy provenance maps in HP; UX changes - oblivian@cumin1003 2426937 wikitext text/x-wiki 628e8fmlln3g9760aby5l2gaxz0bkal 2426938 2426937 2026-06-15T13:26:01Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "Haproxy provenance maps in HP; UX changes - oblivian@cumin1003" 2426938 wikitext text/x-wiki pcyxyo2bs1c8d3hoar154joj6j93zdj 2426939 2426938 2026-06-15T13:28:26Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1080.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2426939 wikitext text/x-wiki 0glpz0qgnqejmk2prvf08de2o0557cb 2426940 2426939 2026-06-15T13:28:33Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on cr2-esams,cr2-esams IPv6 with reason: bouncing pic0 to reconfigure port speeds 2426940 wikitext text/x-wiki 94qss7usbtakdx9wt15dcivongapsot 2426941 2426940 2026-06-15T13:29:48Z Stashbot 7414 topranks: enable BGP graceful-shutdown sender on cr2-esams to drain traffic T427056 2426941 wikitext text/x-wiki 6r5kumzzbl8y4n243ackdlxo3woj6je 2426942 2426941 2026-06-15T13:32:06Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2216: Migration of db2216.codfw.wmnet completed 2426942 wikitext text/x-wiki 55zee500tai7ngnymclghy7ex65mmo7 2426943 2426942 2026-06-15T13:32:09Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2426943 wikitext text/x-wiki bpaihqoi6r5zrhrvbl004k3jbw82dqi 2426944 2426943 2026-06-15T13:34:06Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.reimage for host kafka-logging1006.eqiad.wmnet with OS trixie 2426944 wikitext text/x-wiki tjmqa0fkhnhbhfphauokxeqdw8pebey 2426945 2426944 2026-06-15T13:35:36Z Stashbot 7414 jforrester@deploy1003: Started scap sync-world: Backport for [[gerrit:1300835|Remove no longer used product_metrics.homepage_module_interaction (T365889 T426742)]], [[gerrit:1302153|TaskSuggester: avoid nullable logger in setLogger call]], [[gerrit:1302100|migrateMentorStatusAway: ensure validateStrictly receives objects (T409170)]], [[gerrit:1301451|Store nowiki source in StripState::extra to support subst-nowiki (T3 2426945 wikitext text/x-wiki bou0dshmgoikjmdapkrkp8pvgugs1j7 2426946 2426945 2026-06-15T13:37:19Z Stashbot 7414 jforrester@deploy1003: arlolra, sgimeno, jforrester: Backport for [[gerrit:1300835|Remove no longer used product_metrics.homepage_module_interaction (T365889 T426742)]], [[gerrit:1302153|TaskSuggester: avoid nullable logger in setLogger call]], [[gerrit:1302100|migrateMentorStatusAway: ensure validateStrictly receives objects (T409170)]], [[gerrit:1301451|Store nowiki source in StripState::extra to support subst-nowik 2426946 wikitext text/x-wiki j5n59f97qcluhsgayoqpbz9dlkxzk02 2426948 2426946 2026-06-15T13:39:08Z Stashbot 7414 jforrester@deploy1003: arlolra, sgimeno, jforrester: Continuing with deployment 2426948 wikitext text/x-wiki 9m4ehbbyw84jkr27y6kzl20c3xzmkku 2426950 2426948 2026-06-15T13:40:43Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host conf2007.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2426950 wikitext text/x-wiki ii6xvkcmvicmfh2umh7hgt8nru9c84v 2426951 2426950 2026-06-15T13:42:17Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host conf2007.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2426951 wikitext text/x-wiki 07d8mysui027bx8p5mal6an2xrnsz4e 2426952 2426951 2026-06-15T13:43:28Z Stashbot 7414 jforrester@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300835|Remove no longer used product_metrics.homepage_module_interaction (T365889 T426742)]], [[gerrit:1302153|TaskSuggester: avoid nullable logger in setLogger call]], [[gerrit:1302100|migrateMentorStatusAway: ensure validateStrictly receives objects (T409170)]], [[gerrit:1301451|Store nowiki source in StripState::extra to support subst-nowiki (T 2426952 wikitext text/x-wiki exhvf38vp37qj0lkle1etdea5vhl0qz 2426955 2426952 2026-06-15T13:49:28Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on an-redacteddb1001.eqiad.wmnet with reason: Reboots T426633 2426955 wikitext text/x-wiki ayq7hmmy7q18ba2d2z7m7a7qrbedi0g 2426956 2426955 2026-06-15T13:49:34Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2426956 wikitext text/x-wiki 2v7czla91ca2qc2lxwhio5ahvwmpd0b 2426958 2426956 2026-06-15T13:49:55Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on 11 hosts with reason: Reboots T426633 2426958 wikitext text/x-wiki 6p26pfczr8yeb8vqfs0ce9513qvllv5 2426962 2426958 2026-06-15T13:51:17Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1155.eqiad.wmnet with reason: Reboots T426633 2426962 wikitext text/x-wiki kq9yoz5obu9sri2fbclnfkknwnzjqzb 2426963 2426962 2026-06-15T13:51:45Z Stashbot 7414 fceratto@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1154.eqiad.wmnet with reason: Reboots T426633 2426963 wikitext text/x-wiki b6o46lsnvtrarox87wcs48cx1stmwjy 2426964 2426963 2026-06-15T13:53:58Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver codfw-k8s # T425377: populating translation memory (ttmserver-export.php) on codfw-k8s (dblist: https://phabricator.wikimedia.org/P94145) 2426964 wikitext text/x-wiki 3u2u3sjek4x82wj0s4xedo3x7egweyb 2426965 2426964 2026-06-15T13:54:13Z Stashbot 7414 federico3: doing a quick restart of sanitarium hosts db1155 and db1154 2426965 wikitext text/x-wiki lfl8wid99cynwlhdm7knnagsfc207o0 2426968 2426965 2026-06-15T13:56:18Z Stashbot 7414 elukey@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host kafka-logging1006.eqiad.wmnet with OS trixie 2426968 wikitext text/x-wiki fnykb1yvd0ne1lisz6nuyb1bmgdw6fz 2426969 2426968 2026-06-15T13:56:46Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1077.eqiad.wmnet with OS trixie 2426969 wikitext text/x-wiki ekoqogpclbjm53h1oeuw09kpu7pvrcn 2426976 2426969 2026-06-15T14:00:16Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.major-upgrade 2426976 wikitext text/x-wiki o5jwaxigsbmrlhmxvbqklbtrvpprvxz 2426978 2426976 2026-06-15T14:00:37Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.depool depool db1196: Upgrading db1196.eqiad.wmnet 2426978 wikitext text/x-wiki jz6y5ld1rvy97z67w0wxwpcqmul255d 2426980 2426978 2026-06-15T14:01:27Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1196: Upgrading db1196.eqiad.wmnet 2426980 wikitext text/x-wiki ixfvjtazfbvbjvx8t4lhwaijx55vsjp 2426982 2426980 2026-06-15T14:01:54Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "revert deployment - oblivian@cumin1003" 2426982 wikitext text/x-wiki da5l1ddvyjehcwso1zn6qtb3ak3729m 2426983 2426982 2026-06-15T14:01:56Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: revert deployment - oblivian@cumin1003 2426983 wikitext text/x-wiki 6pv693u6iaf7cw8rtwvon5szzeg6e1a 2426984 2426983 2026-06-15T14:02:49Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: revert deployment - oblivian@cumin1003 2426984 wikitext text/x-wiki q0jwd8fwe2w90slg4jzy2efxy4cv8j9 2426985 2426984 2026-06-15T14:02:51Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "revert deployment - oblivian@cumin1003" 2426985 wikitext text/x-wiki e3lj8lvdf3qn69apm3wt60i9os0wsan 2426986 2426985 2026-06-15T14:03:33Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.reimage for host db1196.eqiad.wmnet with OS trixie 2426986 wikitext text/x-wiki qdbxf9vkbja4zdam1suvkfi732jedo8 2426988 2426986 2026-06-15T14:04:59Z Stashbot 7414 hnowlan@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2426988 wikitext text/x-wiki esfaj6dp1cog9t7mzb6n63328uuleb9 2426989 2426988 2026-06-15T14:05:06Z Stashbot 7414 hnowlan@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2426989 wikitext text/x-wiki 1usxz7l6p5zwpn34hj1wqtifdwnk0ng 2426991 2426989 2026-06-15T14:05:25Z Stashbot 7414 hnowlan@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2426991 wikitext text/x-wiki slzi0bhosfb3k3k0fmisugapk8pgtpq 2426993 2426991 2026-06-15T14:06:47Z Stashbot 7414 hnowlan@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2426993 wikitext text/x-wiki rzu08o8gc5h4rv2e1nmhfusy9aw6mvp 2426994 2426993 2026-06-15T14:07:39Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1077.eqiad.wmnet with reason: host reimage 2426994 wikitext text/x-wiki p31gkc9s7f3vfqciu22ygx3u3mxtx9s 2426995 2426994 2026-06-15T14:07:42Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on cloudvirt1077.eqiad.wmnet with reason: host reimage 2426995 wikitext text/x-wiki 5kgkqkpeokehifxd40fgb2rjs2eak9a 2426996 2426995 2026-06-15T14:07:58Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2426996 wikitext text/x-wiki b3na46kh9l704bymsm6wu11v0n9867o 2426997 2426996 2026-06-15T14:08:30Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2426997 wikitext text/x-wiki sjzwcg4ipd68n2uxswqeyokyzg7df6j 2427003 2426997 2026-06-15T14:17:46Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db1196.eqiad.wmnet with reason: host reimage 2427003 wikitext text/x-wiki 9l91kw5q7c7wedelqp752xp9gqdy0ad 2427008 2427003 2026-06-15T14:23:07Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1196.eqiad.wmnet with reason: host reimage 2427008 wikitext text/x-wiki frmd0cex9zoaj0cutytccbcawz8oymx 2427009 2427008 2026-06-15T14:24:03Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - elukey@cumin1003" 2427009 wikitext text/x-wiki 4vff7nu1wmefyj8xgkjqghw1n8m5wsq 2427011 2427009 2026-06-15T14:24:29Z Stashbot 7414 elukey@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sretest2001.codfw.wmnet with reason: tesT 2427011 wikitext text/x-wiki j5pjn9q7gfn4g9i683c2cj1qio5umys 2427013 2427011 2026-06-15T14:31:10Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - elukey@cumin1003" 2427013 wikitext text/x-wiki ixmxc8tc5dmnrzi2dmoy5mu1rml6fq8 2427014 2427013 2026-06-15T14:31:12Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1077.eqiad.wmnet with OS trixie 2427014 wikitext text/x-wiki 21xo5cp8hc0qczqyxriyl1do91wbt5r 2427016 2427014 2026-06-15T14:41:25Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1196.eqiad.wmnet with OS trixie 2427016 wikitext text/x-wiki bv2iya5d07y8d20k0quemyqd5n1y8ug 2427021 2427016 2026-06-15T14:52:58Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on cr2-esams,cr2-esams IPv6 with reason: bouncing pic0 to reconfigure port speeds 2427021 wikitext text/x-wiki 5ka15vbrmcwbr7jxytv443qvzlnssy4 2427022 2427021 2026-06-15T14:54:09Z Stashbot 7414 topranks: enable BGP graceful-shutdown sender on cr2-esams to drain traffic T427056 2427022 wikitext text/x-wiki eu6q9gzg1o1fux03zck64srguf5gulm 2427024 2427022 2026-06-15T14:55:56Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.mysql.pool pool db1196: Migration of db1196.eqiad.wmnet completed 2427024 wikitext text/x-wiki q1hqbu8lbi8b1untcq05616nvif1qtr 2427025 2427024 2026-06-15T14:57:39Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2427025 wikitext text/x-wiki iduuqqwdb47b6xkyusx6grifgziedly 2427026 2427025 2026-06-15T14:57:44Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host conf2007.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427026 wikitext text/x-wiki c4lz7uqt8zn0qrkwgbbhcr9oxrn3b61 2427027 2427026 2026-06-15T14:58:28Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host conf2007.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427027 wikitext text/x-wiki 5dydksa5dte8sfp41zr7mg99hd8lokm 2427028 2427027 2026-06-15T15:00:50Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2427028 wikitext text/x-wiki s5itxo4pt4p4unetla5e66u0vnh806j 2427029 2427028 2026-06-15T15:01:57Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.admin DNS admin: depool esams [reason: no reason specified, no task ID specified] 2427029 wikitext text/x-wiki 2ucj6uc3c9qleb2add1g5natlxohltt 2427030 2427029 2026-06-15T15:02:44Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool esams [reason: no reason specified, no task ID specified] 2427030 wikitext text/x-wiki 8vsnosfosucnkr36hrziyo7035bj25b 2427031 2427030 2026-06-15T15:02:58Z Stashbot 7414 topranks: depool esams due to cr2-esams rpd crash 2427031 wikitext text/x-wiki fe669v506q9jp0n8konv67slrtluahy 2427033 2427031 2026-06-15T15:13:58Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.dns.admin DNS admin: pool esams [reason: no reason specified, no task ID specified] 2427033 wikitext text/x-wiki canyrm01t7llmr1b3s5ninyb4dfq0yv 2427034 2427033 2026-06-15T15:15:49Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: pool esams [reason: no reason specified, no task ID specified] 2427034 wikitext text/x-wiki est7sfj0hocr7a9jbhb66f51a1br93e 2427035 2427034 2026-06-15T15:16:07Z Stashbot 7414 topranks: repool esams following cr2-esams rpd crash 2427035 wikitext text/x-wiki 9zkmmq0gm6fy0a1yqjsfcxmuemykz7u 2427036 2427035 2026-06-15T15:17:49Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.remove-downtime for an-redacteddb1001.eqiad.wmnet 2427036 wikitext text/x-wiki ds1mytig7yqmt2ynr9d8pdc5vipau4z 2427037 2427036 2026-06-15T15:17:51Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for an-redacteddb1001.eqiad.wmnet 2427037 wikitext text/x-wiki a3t7mi6q11qt9oycs1yepjb6r67ooyx 2427038 2427037 2026-06-15T15:18:03Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.remove-downtime for 11 hosts 2427038 wikitext text/x-wiki i2qatays7zf61c45vq31ytwnfv01p8z 2427039 2427038 2026-06-15T15:18:09Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for 11 hosts 2427039 wikitext text/x-wiki 1i4ybqb4m5wgtdocrmjufy1sb7vgrey 2427040 2427039 2026-06-15T15:19:00Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.remove-downtime for db1154.eqiad.wmnet 2427040 wikitext text/x-wiki giq1kp1dxgrjxahdvvhkwhfwivcyr3o 2427041 2427040 2026-06-15T15:19:02Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for db1154.eqiad.wmnet 2427041 wikitext text/x-wiki dsmnq5kj5fh1t029kycqrr5orkawe14 2427042 2427041 2026-06-15T15:19:06Z Stashbot 7414 cwilliams@cumin1003: START - Cookbook sre.hosts.remove-downtime for db1155.eqiad.wmnet 2427042 wikitext text/x-wiki 2he9nq1q5e84hycmfejnumo8f7n21xw 2427043 2427042 2026-06-15T15:19:08Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for db1155.eqiad.wmnet 2427043 wikitext text/x-wiki fn9ijiasytfs9ze1rfooxqo69n3hn26 2427046 2427043 2026-06-15T15:29:58Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host conf2007.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427046 wikitext text/x-wiki j3mpdktsj64nuwsqf6pos51otliha7h 2427048 2427046 2026-06-15T15:36:54Z Stashbot 7414 dzahn@dns1006: START - running authdns-update 2427048 wikitext text/x-wiki a0q9181d4xhaq07gh3xadsdjb12ozrc 2427049 2427048 2026-06-15T15:40:12Z Stashbot 7414 dzahn@dns1006: END - running authdns-update 2427049 wikitext text/x-wiki gjahhft24ht3gpii58v8b00a2tnwgxq 2427050 2427049 2026-06-15T15:41:05Z Stashbot 7414 mutante: added new project language 'nyn' - Bantu language spoken by the Nkore and Hema peoples of Southwestern Uganda 2427050 wikitext text/x-wiki ohygrs2f4tc4f2ehevabsz7844mkdb4 2427051 2427050 2026-06-15T15:41:26Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1196: Migration of db1196.eqiad.wmnet completed 2427051 wikitext text/x-wiki 2d259xqko1f81x81daqaj1z94qaci97 2427052 2427051 2026-06-15T15:41:28Z Stashbot 7414 cwilliams@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2427052 wikitext text/x-wiki 8v36c9qil5pqc4wba7ulfusid5pwxzc 2427053 2427052 2026-06-15T15:49:09Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host conf2007.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427053 wikitext text/x-wiki c8qvg1ou7toedofzs3venn486nomtm0 2427054 2427053 2026-06-15T15:50:21Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host conf2008.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427054 wikitext text/x-wiki cekj7spw40owr0ujgbzwi8xdc40j9jd 2427055 2427054 2026-06-15T15:50:48Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on releases1003.eqiad.wmnet with reason: puppet debugging 2427055 wikitext text/x-wiki 3f9g3xxkodbaov3gj25c5nr9vy6w30p 2427056 2427055 2026-06-15T15:51:15Z Stashbot 7414 dzahn@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on releases2003.codfw.wmnet with reason: puppet debugging 2427056 wikitext text/x-wiki 5vs7gxxbx71vd7qun6ud50weqwu1ggn 2427057 2427056 2026-06-15T15:51:34Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host conf2008.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427057 wikitext text/x-wiki lw6ea6rksusa0i3fce6h6hmutp2ju2w 2427058 2427057 2026-06-15T15:57:46Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host conf2008.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427058 wikitext text/x-wiki m7hrfeqcitn0i7npltqu0qc9exzm537 2427059 2427058 2026-06-15T16:04:10Z Stashbot 7414 elukey@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host conf2008.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427059 wikitext text/x-wiki lb1qfagerv8i0f5asrwgcs3ewlo5fqw 2427060 2427059 2026-06-15T16:04:40Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.hosts.provision for host conf2008.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427060 wikitext text/x-wiki b15fxw4a9fsw8vawmoi3umecl3szi29 2427061 2427060 2026-06-15T16:05:41Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1302192|SourceEditorOverlayHookPayload: Allow aborting of the save (T428287)]], [[gerrit:1302194|hCaptcha MobileFrontend: Avoid indefinite save loop on known errors (T428287)]], [[gerrit:1302195|OATHUserRepository: Specify caller in query]], [[gerrit:1302186|Bump guzzlehttp/psr to version 2.11.0 (T429208)]], [[gerrit:1302169|NoReferrerLinks: Add rel 2427061 wikitext text/x-wiki 79krs2bf1u65yjgycimu68gx1tfqsy7 2427062 2427061 2026-06-15T16:07:27Z Stashbot 7414 dreamyjazz@deploy1003: reedy, dreamyjazz, kharlan: Backport for [[gerrit:1302192|SourceEditorOverlayHookPayload: Allow aborting of the save (T428287)]], [[gerrit:1302194|hCaptcha MobileFrontend: Avoid indefinite save loop on known errors (T428287)]], [[gerrit:1302195|OATHUserRepository: Specify caller in query]], [[gerrit:1302186|Bump guzzlehttp/psr to version 2.11.0 (T429208)]], [[gerrit:1302169|NoReferrerLinks: Add 2427062 wikitext text/x-wiki ppzj0129ha6mblj4trx0j0l6ek0n38h 2427063 2427062 2026-06-15T16:08:39Z Stashbot 7414 atsuko@deploy1003: helmfile [eqiad] START helmfile.d/services/toolhub: apply 2427063 wikitext text/x-wiki o1r4r4jdufym07m7hhxd5p5zrgh7syi 2427064 2427063 2026-06-15T16:08:53Z Stashbot 7414 dreamyjazz@deploy1003: reedy, dreamyjazz, kharlan: Continuing with deployment 2427064 wikitext text/x-wiki hclq3w7lkurtq09fgoi3sxg4edykqon 2427065 2427064 2026-06-15T16:08:59Z Stashbot 7414 atsuko@deploy1003: helmfile [eqiad] DONE helmfile.d/services/toolhub: apply 2427065 wikitext text/x-wiki 5hlul5cwwfr3n1f024ux54nrq1kvhve 2427066 2427065 2026-06-15T16:10:46Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host conf2008.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427066 wikitext text/x-wiki ilmxysz53hidzi3rycilev5pn4rl5vo 2427067 2427066 2026-06-15T16:13:05Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2427067 wikitext text/x-wiki 2todvvu40aq7yp24m4ml9pbdrch5tyg 2427068 2427067 2026-06-15T16:13:18Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1302192|SourceEditorOverlayHookPayload: Allow aborting of the save (T428287)]], [[gerrit:1302194|hCaptcha MobileFrontend: Avoid indefinite save loop on known errors (T428287)]], [[gerrit:1302195|OATHUserRepository: Specify caller in query]], [[gerrit:1302186|Bump guzzlehttp/psr to version 2.11.0 (T429208)]], [[gerrit:1302169|NoReferrerLinks: Add re 2427068 wikitext text/x-wiki 083xq70yvxb3pfkz74vde65qoonmk86 2427069 2427068 2026-06-15T16:16:06Z Stashbot 7414 atsuko@deploy1003: helmfile [codfw] START helmfile.d/services/toolhub: apply 2427069 wikitext text/x-wiki gut1890ejhqmghh9tilzyzhetxeg6rs 2427070 2427069 2026-06-15T16:16:35Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2427070 wikitext text/x-wiki d8aryun7c9czjoyyg3ddnx5vs8ubunz 2427071 2427070 2026-06-15T16:16:46Z Stashbot 7414 atsuko@deploy1003: helmfile [codfw] DONE helmfile.d/services/toolhub: apply 2427071 wikitext text/x-wiki lj1i4fycqqlj74qdrgvr9u2ob9o7csm 2427074 2427071 2026-06-15T16:36:03Z Stashbot 7414 atsuko@deploy1003: helmfile [staging] START helmfile.d/services/toolhub: apply 2427074 wikitext text/x-wiki n5o66qeibbpsumk2zq71wx7a6u6405j 2427075 2427074 2026-06-15T16:36:13Z Stashbot 7414 atsuko@deploy1003: helmfile [staging] DONE helmfile.d/services/toolhub: apply 2427075 wikitext text/x-wiki b8h6f4mw1r7aeb6vgc76gqt3egqk629 2427078 2427075 2026-06-15T17:03:48Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2427078 wikitext text/x-wiki ifxakgtvorzrbff8iaok8f0o9qmxta8 2427081 2427078 2026-06-15T17:07:35Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.reimage for host cp5026.eqsin.wmnet with OS trixie 2427081 wikitext text/x-wiki 4p87sga23r018py5z31pamih0zwgbpz 2427082 2427081 2026-06-15T17:09:48Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.move-vlan for host cp5026 2427082 wikitext text/x-wiki 4xbvcyypvtgkqax5fv4l2dhbrw2jcpn 2427089 2427082 2026-06-15T17:34:27Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.network.tls for network device lsw1-c4-eqiad 2427089 wikitext text/x-wiki 00veb1o1xt1ptyy5zwrhgfc701t4pmn 2427090 2427089 2026-06-15T17:34:30Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-c4-eqiad 2427090 wikitext text/x-wiki bgjnqel2216n8cxg3en6fqo0k89k68b 2427091 2427090 2026-06-15T17:36:00Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.network.tls for network device lsw1-c4-eqiad 2427091 wikitext text/x-wiki 2ta9xnw4jkevsxd71cydomr9imryns8 2427092 2427091 2026-06-15T17:36:02Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-c4-eqiad 2427092 wikitext text/x-wiki ph0vxphx1bgwt8mklsq5h7c0ghutgt9 2427093 2427092 2026-06-15T17:40:27Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.network.tls for network device ssw1-d8-eqiad 2427093 wikitext text/x-wiki 98h5h0a86beqp0x8ots98xjt9rny108 2427094 2427093 2026-06-15T17:40:29Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device ssw1-d8-eqiad 2427094 wikitext text/x-wiki suxdxfobfmawusgv0pqcsrrqwc1x0c3 2427095 2427094 2026-06-15T17:46:35Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.dns.netbox 2427095 wikitext text/x-wiki fs5osidlkbjem96smfhtjrc9n0649oc 2427096 2427095 2026-06-15T17:52:16Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host cp5026 - brett@cumin2002" 2427096 wikitext text/x-wiki 2ht3zkjsv23f2mb44qt0q7oex99h2tj 2427097 2427096 2026-06-15T17:52:26Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host cp5026 - brett@cumin2002" 2427097 wikitext text/x-wiki 42w12ci8t0fr2vrkzcuc678q1f0ex24 2427098 2427097 2026-06-15T17:52:27Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2427098 wikitext text/x-wiki dk3e3n02eqilwdal5pt9nxhy1gozs34 2427099 2427098 2026-06-15T17:52:29Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.dns.wipe-cache cp5026.eqsin.wmnet 37.0.132.10.in-addr.arpa 7.3.0.0.0.0.0.0.2.3.1.0.0.1.0.0.1.0.1.0.0.0.5.e.2.f.d.0.1.0.0.2.ip6.arpa on all recursors 2427099 wikitext text/x-wiki d26m56muxjvep6gvb9onbl9suwwrgg5 2427100 2427099 2026-06-15T17:52:31Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) cp5026.eqsin.wmnet 37.0.132.10.in-addr.arpa 7.3.0.0.0.0.0.0.2.3.1.0.0.1.0.0.1.0.1.0.0.0.5.e.2.f.d.0.1.0.0.2.ip6.arpa on all recursors 2427100 wikitext text/x-wiki r9qk18j4123l5jprrfucs7jdedlbzsu 2427101 2427100 2026-06-15T17:52:34Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host cp5026 2427101 wikitext text/x-wiki h7km41l642zjszsui9ogqlp9y4xnewn 2427102 2427101 2026-06-15T17:53:20Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cp5026 2427102 wikitext text/x-wiki 5dcp99l5lyrckpmcxe0zqm4zwjg3dgr 2427103 2427102 2026-06-15T17:53:22Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host cp5026 2427103 wikitext text/x-wiki pyo850tnt2kmlsxcou48b9phhp3c0ax 2427104 2427103 2026-06-15T18:18:05Z Stashbot 7414 mutante: releases2003 - systemctl stop tmp.mount 2427104 wikitext text/x-wiki 0b9wv6aee2sbybuq7jw1ezmab906hnx 2427107 2427104 2026-06-15T18:27:27Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cp5026.eqsin.wmnet with reason: host reimage 2427107 wikitext text/x-wiki 7rjzboep50ihj5vjb4mtabjpm0ha1mx 2427108 2427107 2026-06-15T18:27:29Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.loadbalancer.admin config_reloading P{lvs5005.eqsin.wmnet} and A:liberica 2427108 wikitext text/x-wiki mithnx18bqm2dl67qwa4a47y7gs686r 2427109 2427108 2026-06-15T18:27:53Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) config_reloading P{lvs5005.eqsin.wmnet} and A:liberica 2427109 wikitext text/x-wiki kxdp8wfyzt66akuyznskd89nhq6shar 2427110 2427109 2026-06-15T18:35:01Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp5026.eqsin.wmnet with reason: host reimage 2427110 wikitext text/x-wiki 07o7enc8wd5plim0cd8wiaqz31nlqac 2427112 2427110 2026-06-15T18:42:24Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.cdn.roll-restart-purged rolling restart_daemons on P{cp7001.magru.wmnet} and A:cp 2427112 wikitext text/x-wiki l7fhjmts9vsojuqqotbpglbfum0g1yi 2427114 2427112 2026-06-15T18:44:03Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.cdn.roll-restart-purged (exit_code=0) rolling restart_daemons on P{cp7001.magru.wmnet} and A:cp 2427114 wikitext text/x-wiki fveh5t39r7j8pirelm8j10grj7vz79d 2427119 2427114 2026-06-15T19:04:07Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp5026.eqsin.wmnet with OS trixie 2427119 wikitext text/x-wiki ge9f5ioh17wqyvasr10r92aafmwupnv 2427120 2427119 2026-06-15T19:04:50Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.loadbalancer.admin config_reloading P{lvs5005.eqsin.wmnet} and A:liberica 2427120 wikitext text/x-wiki s7jn60jmaavnwe4vn9lsqahpd2cw46g 2427121 2427120 2026-06-15T19:05:03Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) config_reloading P{lvs5005.eqsin.wmnet} and A:liberica 2427121 wikitext text/x-wiki fn7fa4ks6xfr4n45kxpgjw6zedhhxhz 2427122 2427121 2026-06-15T19:05:24Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp5026.* 2427122 wikitext text/x-wiki plnqnd6qluoor5sbhw57dhkdlh7ak5q 2427123 2427122 2026-06-15T19:06:52Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=no; selector: name=cp5026.* 2427123 wikitext text/x-wiki e5aps157c2niu566g7d6x9man0sz305 2427125 2427123 2026-06-15T19:14:56Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.loadbalancer.admin config_reloading P{lvs5004.eqsin.wmnet} and A:liberica 2427125 wikitext text/x-wiki lkyt3cf355y2c7kjct61hvbotvjfpui 2427126 2427125 2026-06-15T19:15:20Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) config_reloading P{lvs5004.eqsin.wmnet} and A:liberica 2427126 wikitext text/x-wiki p4z9a2k8ftgr0lkda5fhloxo6nkpv8v 2427127 2427126 2026-06-15T19:16:54Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.loadbalancer.upgrade restart P{lvs5005.eqsin.wmnet} and A:liberica 2427127 wikitext text/x-wiki igx1foa740ke26jkhjd3hyp902pxttc 2427128 2427127 2026-06-15T19:17:53Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.loadbalancer.upgrade (exit_code=0) restart P{lvs5005.eqsin.wmnet} and A:liberica 2427128 wikitext text/x-wiki edo4t2xu31alqmgwshvfgzxoufa0zke 2427129 2427128 2026-06-15T19:18:20Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp5026.* 2427129 wikitext text/x-wiki kim4cle0ms9ty3fxcniu98fe7z3azh0 2427130 2427129 2026-06-15T19:21:08Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.loadbalancer.upgrade restart A:liberica-eqsin 2427130 wikitext text/x-wiki too8efkkw4hwkejuk3wtp0e1ia3xgbi 2427131 2427130 2026-06-15T19:23:56Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.loadbalancer.upgrade (exit_code=0) restart A:liberica-eqsin 2427131 wikitext text/x-wiki nzlod24uj06ou93dtghlrcpezwcam2p 2427132 2427131 2026-06-15T19:25:54Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.reimage for host cp5028.eqsin.wmnet with OS trixie 2427132 wikitext text/x-wiki eyitbo26yrvbgtjzupn53bf4grw9jia 2427133 2427132 2026-06-15T19:26:25Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.move-vlan for host cp5028 2427133 wikitext text/x-wiki n99no5yc2u2co0m9yc10mln4ard1ct2 2427135 2427133 2026-06-15T19:33:07Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=no; selector: name=cp3066.esams.wmnet 2427135 wikitext text/x-wiki d7s0n53yf88krfwvcyh6b6zbmzii0et 2427136 2427135 2026-06-15T19:33:12Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2001.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427136 wikitext text/x-wiki mznam7xzzlgxhcey0pxjff04d1396so 2427137 2427136 2026-06-15T19:33:57Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp3066.esams.wmnet 2427137 wikitext text/x-wiki 5q1vc2ubacmudpgpkcj1a9y9o3azn4o 2427138 2427137 2026-06-15T19:34:00Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp5026.* 2427138 wikitext text/x-wiki tt43gsb3ekqhurhn78t645dc458xcqy 2427139 2427138 2026-06-15T19:34:29Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=no; selector: name=cp3067.esams.wmnet 2427139 wikitext text/x-wiki erxqzuv4305vdipwqvs5dwvxmn4k2eg 2427140 2427139 2026-06-15T19:35:02Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp3067.esams.wmnet 2427140 wikitext text/x-wiki hb4xjh3jks0282i66uyxnqemtfh3eqv 2427141 2427140 2026-06-15T19:36:21Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.dns.netbox 2427141 wikitext text/x-wiki mt9cdgwhb49swpseo3i4svrblvnwlg7 2427142 2427141 2026-06-15T19:40:12Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dse-k8s-wdqs2001.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART 2427142 wikitext text/x-wiki fyp9qcv39aquwf55565wjju8xsnrn8w 2427143 2427142 2026-06-15T19:42:56Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host cp5028 - brett@cumin2002" 2427143 wikitext text/x-wiki 1ppr69fdvjc4neqtuotrgcxp27ic5k5 2427144 2427143 2026-06-15T19:43:02Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host cp5028 - brett@cumin2002" 2427144 wikitext text/x-wiki tlvhx58v1ux2s4c99mv6zcjfhe4nd4q 2427145 2427144 2026-06-15T19:43:04Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2427145 wikitext text/x-wiki jrw7ij4q43i9igo39hanrpki2slcgaf 2427146 2427145 2026-06-15T19:43:06Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.dns.wipe-cache cp5028.eqsin.wmnet 25.0.132.10.in-addr.arpa 5.2.0.0.0.0.0.0.2.3.1.0.0.1.0.0.1.0.1.0.0.0.5.e.2.f.d.0.1.0.0.2.ip6.arpa on all recursors 2427146 wikitext text/x-wiki 0zm7k1i4jf8qvslb9xe9uesthbih644 2427147 2427146 2026-06-15T19:43:08Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) cp5028.eqsin.wmnet 25.0.132.10.in-addr.arpa 5.2.0.0.0.0.0.0.2.3.1.0.0.1.0.0.1.0.1.0.0.0.5.e.2.f.d.0.1.0.0.2.ip6.arpa on all recursors 2427147 wikitext text/x-wiki gvb7r4u1pgb6m7tz649032uj77pztnn 2427148 2427147 2026-06-15T19:43:10Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host cp5028 2427148 wikitext text/x-wiki jxlufhz1ycp3xdgbavxm1n2ew6kd5gn 2427149 2427148 2026-06-15T19:44:03Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cp5028 2427149 wikitext text/x-wiki ckiwwb60zuqs1a92yrbblewd5moae8j 2427150 2427149 2026-06-15T19:44:05Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host cp5028 2427150 wikitext text/x-wiki jjip0yr4ql166gqijhgdo4m42m1xjv6 2427151 2427150 2026-06-15T19:44:29Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host dse-k8s-wdqs2001.codfw.wmnet with OS trixie 2427151 wikitext text/x-wiki qvdm99cf9sj4rusgn879zkgpeez7v5h 2427156 2427151 2026-06-15T20:02:52Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host dse-k8s-wdqs2001.codfw.wmnet with OS trixie 2427156 wikitext text/x-wiki 03wxk7gi3kcc47n196oknzb7s33gbdk 2427157 2427156 2026-06-15T20:10:10Z Stashbot 7414 dancy@deploy1003: Started scap sync-world: Backport for [[gerrit:1300245|REST: set new RestModuleOverrides variable (T422756)]], [[gerrit:1302232|Enable "exit the editor" survey on 11 wikis for phase 2 (T426132)]] 2427157 wikitext text/x-wiki oc8r6hslnhn24ikbd6pivwzgymw0i4c 2427158 2427157 2026-06-15T20:14:13Z Stashbot 7414 dancy@deploy1003: caro, dancy, bpirkle: Backport for [[gerrit:1300245|REST: set new RestModuleOverrides variable (T422756)]], [[gerrit:1302232|Enable "exit the editor" survey on 11 wikis for phase 2 (T426132)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2427158 wikitext text/x-wiki kdks23sjqcwphsmt4ihf2o8u0elv8zx 2427159 2427158 2026-06-15T20:16:45Z Stashbot 7414 dancy@deploy1003: caro, dancy, bpirkle: Continuing with deployment 2427159 wikitext text/x-wiki sqytgn4907xasuaq3swcnbn48dkb3wt 2427160 2427159 2026-06-15T20:17:03Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on cp5028.eqsin.wmnet with reason: host reimage 2427160 wikitext text/x-wiki 4ee3ccb96xfjw5y0jy6jnca33kwnksj 2427161 2427160 2026-06-15T20:21:04Z Stashbot 7414 dancy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1300245|REST: set new RestModuleOverrides variable (T422756)]], [[gerrit:1302232|Enable "exit the editor" survey on 11 wikis for phase 2 (T426132)]] (duration: 10m 54s) 2427161 wikitext text/x-wiki tmxguca0yuiqcscc77vc9gtpe8ccjxs 2427162 2427161 2026-06-15T20:24:31Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp5028.eqsin.wmnet with reason: host reimage 2427162 wikitext text/x-wiki 6t64kllojwj6g2g8tcw5eozid2e51n8 2427166 2427162 2026-06-15T20:52:26Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp5028.eqsin.wmnet with OS trixie 2427166 wikitext text/x-wiki hevf8y85hmkdgxijd6frhxvgb04bass 2427171 2427166 2026-06-15T21:05:11Z Stashbot 7414 brett@cumin2002: START - Cookbook sre.loadbalancer.upgrade restart P{lvs5005.eqsin.wmnet} and A:liberica 2427171 wikitext text/x-wiki f541r9koo0f3vhhpbvzcuccejoqnwtk 2427172 2427171 2026-06-15T21:06:10Z Stashbot 7414 brett@cumin2002: END (PASS) - Cookbook sre.loadbalancer.upgrade (exit_code=0) restart P{lvs5005.eqsin.wmnet} and A:liberica 2427172 wikitext text/x-wiki 8l2zbd7msp662yofzqfr8xl7ypy82ne 2427173 2427172 2026-06-15T21:06:29Z Stashbot 7414 brett@puppetserver1001: conftool action : set/pooled=yes; selector: name=cp5028.* 2427173 wikitext text/x-wiki qa4557esr2u5c0l6fhohq93nmwd1kkt 2427174 2427173 2026-06-15T21:13:55Z Stashbot 7414 sbassett@deploy1003: Started scap sync-world: Backport for [[gerrit:1302267|ForceReauth: Avoid unnecessary securitySensitiveOperationStatus checks]] 2427174 wikitext text/x-wiki nmpzql9f9j3leq19wrp45768mndxz1y 2427175 2427174 2026-06-15T21:15:40Z Stashbot 7414 sbassett@deploy1003: sbassett: Backport for [[gerrit:1302267|ForceReauth: Avoid unnecessary securitySensitiveOperationStatus checks]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2427175 wikitext text/x-wiki pbnphfhywervhsc5bi4x4tg31bbp5u7 2427176 2427175 2026-06-15T21:17:51Z Stashbot 7414 sbassett@deploy1003: sbassett: Continuing with deployment 2427176 wikitext text/x-wiki 36pzimbikoepndtly4jt7bxig5en6y8 2427177 2427176 2026-06-15T21:22:06Z Stashbot 7414 sbassett@deploy1003: Finished scap sync-world: Backport for [[gerrit:1302267|ForceReauth: Avoid unnecessary securitySensitiveOperationStatus checks]] (duration: 08m 11s) 2427177 wikitext text/x-wiki c0w0nxnfka3gd9858u5cvve6zpi901z 2427178 2427177 2026-06-15T21:40:26Z Stashbot 7414 sbassett: Deployed security fix for T428820 2427178 wikitext text/x-wiki t073zxbvgg1vk2byt3j0itwtoke96x8 2427179 2427178 2026-06-15T21:48:32Z Stashbot 7414 dancy@deploy1003: Started scap sync-world: Backport for [[gerrit:1302277|beta: Point remaining db11 references at deployment-db15 (T428930)]] 2427179 wikitext text/x-wiki rpzrpw3ndtgctxoziqfo6gy3kxaoaen 2427181 2427179 2026-06-15T21:48:59Z Stashbot 7414 sbassett: Deployed security fix for T428809 2427181 wikitext text/x-wiki mm5sjfxkom2ont13qixxtt6e62uc51f 2427182 2427181 2026-06-15T21:49:25Z Stashbot 7414 dancy@deploy1003: dancy: Backport for [[gerrit:1302277|beta: Point remaining db11 references at deployment-db15 (T428930)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2427182 wikitext text/x-wiki cre76parth0xg4gx3oiei8h1qw9q8cn 2427183 2427182 2026-06-15T21:53:22Z Stashbot 7414 dancy@deploy1003: dancy: Continuing with deployment 2427183 wikitext text/x-wiki k9dz2yrwguwtbdzhyckd6ts35d3wfg9 2427184 2427183 2026-06-15T21:54:58Z Stashbot 7414 dancy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1302277|beta: Point remaining db11 references at deployment-db15 (T428930)]] (duration: 12m 27s) 2427184 wikitext text/x-wiki fupteipfy9qnwuk568ixqpgyq7zv38x 2427185 2427184 2026-06-15T22:03:37Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2427185 wikitext text/x-wiki 29wogwiqpmst1w4v6o2fockgy5519ji 2427186 2427185 2026-06-15T22:04:20Z Stashbot 7414 arlolra@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2427186 wikitext text/x-wiki k75jp8sotd836jyidpitetekouu1bjb 2427187 2427186 2026-06-15T22:04:22Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2427187 wikitext text/x-wiki ia90qsxf9rpzct4edfzzn8pgoz4qj4u 2427188 2427187 2026-06-15T22:04:52Z Stashbot 7414 arlolra@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2427188 wikitext text/x-wiki 5m4gt0hls6hv59g04jk34mzao35fgoq 2427189 2427188 2026-06-15T22:05:04Z Stashbot 7414 sbassett: Deployed updated security fix for T427611 2427189 wikitext text/x-wiki 40hwg9vdb2bnezolvl0la1ktwczsn9u 2427190 2427189 2026-06-15T22:35:46Z Stashbot 7414 sbassett: Deployed private config for T429244 2427190 wikitext text/x-wiki 12l1i132q6ub1ksgfvz9jbhmxjclsx2 2427199 2427190 2026-06-16T04:05:31Z Stashbot 7414 mwpresync@deploy1003: Pruned MediaWiki: 1.47.0-wmf.4 (duration: 05m 29s) 2427199 wikitext text/x-wiki kpgf227i47ge26axwp7nv8rzsdc4snn 2427200 2427199 2026-06-16T04:55:03Z Stashbot 7414 ryankemper: T427951 Deleted 4 leftover mirrored dev/test topics from kafka-test: `eqiad.mediawiki.{page_html_content_change.dev{1,4},page_edit_type_simple.dev0}`, `eqiad.mw_page_edit_type_enrich.error` 2427200 wikitext text/x-wiki 4si4pmzpkgm7dqbede2jgsc4vjshdbk 2427201 2427200 2026-06-16T05:58:48Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2427201 wikitext text/x-wiki c1qsh63xzfnd4oi9bb7fs55hbn269a4 2427202 2427201 2026-06-16T05:59:09Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es1047: Upgrading es1047.eqiad.wmnet 2427202 wikitext text/x-wiki hg8xnmbt4dogj16zvjy26wu0o7gdv7d 2427205 2427202 2026-06-16T06:24:34Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.depool (exit_code=99) depool es1047: Upgrading es1047.eqiad.wmnet 2427205 wikitext text/x-wiki 1qkd57wcr98o1l2fw3ppmc0slur4vb2 2427206 2427205 2026-06-16T06:24:37Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2427206 wikitext text/x-wiki rvxi0sjcv74ggnyg1zp3ngzuq58bi5k 2427207 2427206 2026-06-16T06:24:45Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2427207 wikitext text/x-wiki qaxh1iuvfthvlj7okjluryn8ahh9zhv 2427208 2427207 2026-06-16T06:24:48Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.mysql.major-upgrade (exit_code=99) 2427208 wikitext text/x-wiki rjhiajg7imc7f3s72agup118x1audmm 2427209 2427208 2026-06-16T06:25:36Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es1047.eqiad.wmnet with OS trixie 2427209 wikitext text/x-wiki j2630c4zucyjle56opfdgx17awtfz52 2427210 2427209 2026-06-16T06:40:30Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es1047.eqiad.wmnet with reason: host reimage 2427210 wikitext text/x-wiki 8tqbmii4gytj84grae2agud3js5shxg 2427211 2427210 2026-06-16T06:44:18Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es1047.eqiad.wmnet with reason: host reimage 2427211 wikitext text/x-wiki ckz50wzltq5msrw9owdnuyu1c74q9y1 2427212 2427211 2026-06-16T06:50:35Z Stashbot 7414 hashar@deploy1003: Started deploy [integration/docroot@2165507]: build: Updating js-yaml to 4.2.0 2427212 wikitext text/x-wiki t7ra44falmlqbh013yiw3x9ri1t1qx0 2427213 2427212 2026-06-16T06:50:52Z Stashbot 7414 hashar@deploy1003: Finished deploy [integration/docroot@2165507]: build: Updating js-yaml to 4.2.0 (duration: 00m 16s) 2427213 wikitext text/x-wiki ksfzrovwrrng933i4v7yhltutola77z 2427214 2427213 2026-06-16T07:08:02Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es1047.eqiad.wmnet with OS trixie 2427214 wikitext text/x-wiki a0cyktcenoqmur6serno4utv4lcr65o 2427215 2427214 2026-06-16T07:14:07Z Stashbot 7414 wmde-fisch@deploy1003: Started scap sync-world: Backport for [[gerrit:1302170|Update VE core submodule to master (3e79e9934) (T397319 T428764)]] 2427215 wikitext text/x-wiki 7gz45t1icfbpgo5d5oevakh6luyj1fw 2427216 2427215 2026-06-16T07:33:48Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch: Backport for [[gerrit:1302170|Update VE core submodule to master (3e79e9934) (T397319 T428764)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2427216 wikitext text/x-wiki af3eyqb4mnp51z78dodva5zfmb33pyx 2427217 2427216 2026-06-16T07:36:09Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch: Continuing with deployment 2427217 wikitext text/x-wiki s2veg1chj526ft1t0a2qkvvfjam93gh 2427218 2427217 2026-06-16T07:50:20Z Stashbot 7414 wmde-fisch@deploy1003: Finished scap sync-world: Backport for [[gerrit:1302170|Update VE core submodule to master (3e79e9934) (T397319 T428764)]] (duration: 36m 13s) 2427218 wikitext text/x-wiki stbckstovcd8jlqow5ea8dikirjsgdg 2427219 2427218 2026-06-16T07:54:01Z Stashbot 7414 wmde-fisch@deploy1003: Started scap sync-world: Backport for [[gerrit:1302629|Improve click intent event logging and exposure tracking]] 2427219 wikitext text/x-wiki snl3lk7vj1wn217lcin4yqym1bvu9cr 2427221 2427219 2026-06-16T07:58:02Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es1047: repool after upgrade 2427221 wikitext text/x-wiki rwzasu4a5dg7o4rjytlf0gkp3e8igqx 2427222 2427221 2026-06-16T07:58:17Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch: Backport for [[gerrit:1302629|Improve click intent event logging and exposure tracking]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2427222 wikitext text/x-wiki mmqz183jj0e9q3o829slgmzdq0t5dv5 2427223 2427222 2026-06-16T07:58:42Z Stashbot 7414 wmde-fisch@deploy1003: wmde-fisch: Continuing with deployment 2427223 wikitext text/x-wiki 8ucr24fhlqo4a4lgmgq8fkgu4cps3dk 2427224 2427223 2026-06-16T08:00:42Z Stashbot 7414 moritzm: update bird on ganeti7001 to 2.18.2-1~wmf12u1 2427224 wikitext text/x-wiki ams2d0b6q4vs33yi7r6459uc8qjft33 2427225 2427224 2026-06-16T08:05:32Z Stashbot 7414 wmde-fisch@deploy1003: Finished scap sync-world: Backport for [[gerrit:1302629|Improve click intent event logging and exposure tracking]] (duration: 11m 31s) 2427225 wikitext text/x-wiki 9oavwx16ac0qwt2mpvo0fhv5v1abxr5 2427226 2427225 2026-06-16T08:17:31Z Stashbot 7414 atsuko@deploy1003: mwscript-k8s job started: foreachwikiindblist mwscript.dblist extensions/Translate/scripts/ttmserver-export.php --ttmserver codfw-k8s # T425377: populating translation memory (ttmserver-export.php) on codfw-k8s (dblist: https://phabricator.wikimedia.org/P94157) 2427226 wikitext text/x-wiki 7lw3rhcobqpf4vx1xb87paz9c5bfq59 2427227 2427226 2026-06-16T08:19:51Z Stashbot 7414 mszwarc@deploy1003: Synchronized private/SuggestedInvestigationsSignals: Private code deployment for Suggested Investigations (duration: 06m 03s) 2427227 wikitext text/x-wiki hldshug4g8xr9kp5dt6fscplj5mjpgi 2427228 2427227 2026-06-16T08:23:10Z Stashbot 7414 mszwarc@deploy1003: Synchronized private/PrivateSettings.php: Private code deployment for Suggested Investigations (duration: 02m 23s) 2427228 wikitext text/x-wiki fjmctol5yl837z59vbx0f7czucg7n8a 2427229 2427228 2026-06-16T08:27:23Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1302735|hCaptcha: Enable for MobileFrontend in all wikis (T425940)]] 2427229 wikitext text/x-wiki pcezvctro1gw5zk5xryynt0ffe7tz3q 2427230 2427229 2026-06-16T08:29:21Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1302735|hCaptcha: Enable for MobileFrontend in all wikis (T425940)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2427230 wikitext text/x-wiki 2jpho658l7cokmh7ecfgysudy3lppdn 2427231 2427230 2026-06-16T08:32:55Z Stashbot 7414 moritzm: installing nginx security updates 2427231 wikitext text/x-wiki kbfikot3bruriib8rlb5nr8dg7mqbsd 2427233 2427231 2026-06-16T08:42:27Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2427233 wikitext text/x-wiki clnb44n13qqnhy7asgaxfwk843hlmlz 2427234 2427233 2026-06-16T08:43:26Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es1047: repool after upgrade 2427234 wikitext text/x-wiki 4b0nh5kcmjlzrik5xwzhsbi9pv0dok1 2427235 2427234 2026-06-16T08:45:37Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2427235 wikitext text/x-wiki 9x2pkpnr8tihwae0deetbmg0pox0fh4 2427236 2427235 2026-06-16T08:45:58Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es1036: Upgrading es1036.eqiad.wmnet 2427236 wikitext text/x-wiki erg1tnab0v4cu79nj6mxvz6otzmux2o 2427239 2427236 2026-06-16T08:46:46Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1302735|hCaptcha: Enable for MobileFrontend in all wikis (T425940)]] (duration: 19m 23s) 2427239 wikitext text/x-wiki 65rkqa01qvhdfidhg20efbqpesjgi5g 2427241 2427239 2026-06-16T08:47:59Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es1036: Upgrading es1036.eqiad.wmnet 2427241 wikitext text/x-wiki 9w88ca0n5ggsvr5qzu32mx3tcmjx0qc 2427242 2427241 2026-06-16T08:48:39Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es1036.eqiad.wmnet with OS trixie 2427242 wikitext text/x-wiki nuge3y046jdnd5185a131guz3bj20kx 2427244 2427242 2026-06-16T08:56:26Z Stashbot 7414 moritzm: uploaded bird 2.18.2-1~wmf12u1 to bookworm-wikimedia T429285 2427244 wikitext text/x-wiki g6xrw6i90quvhnrrpjpmly87vkr9ps6 2427245 2427244 2026-06-16T09:00:43Z Stashbot 7414 urbanecm@deploy1003: mwscript-k8s job started: foreachwikiindblist wikidata WikimediaMaintenance:createExtensionTables.php GrowthExperiments # T418115 2427245 wikitext text/x-wiki c5n01cir3f9hrwfppl3tradfgu8cggo 2427246 2427245 2026-06-16T09:01:04Z Stashbot 7414 moritzm: uploaded bird 2.18.2-1~wmf13u1 to trixie-wikimedia T429285 2427246 wikitext text/x-wiki qf92oawdm8n8pjiodbsnbl40crs96vk 2427247 2427246 2026-06-16T09:02:48Z Stashbot 7414 urbanecm@deploy1003: Started scap sync-world: Backport for [[gerrit:1297161|[Growth] wikidatawiki: Enable Growth features (T418115)]] 2427247 wikitext text/x-wiki 5qglsskcy7xp8a9eyzurdgmkbay0spq 2427248 2427247 2026-06-16T09:04:16Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es1036.eqiad.wmnet with reason: host reimage 2427248 wikitext text/x-wiki o8wrerq96agwxa1gadrqpzt2eemo57t 2427249 2427248 2026-06-16T09:04:50Z Stashbot 7414 urbanecm@deploy1003: urbanecm: Backport for [[gerrit:1297161|[Growth] wikidatawiki: Enable Growth features (T418115)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2427249 wikitext text/x-wiki iddg0ys9fp8y37hhfkd1hcfgepnf8lp 2427250 2427249 2026-06-16T09:06:05Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2427250 wikitext text/x-wiki 85r4qt8yjjcczamup96y9u95uo4qgg6 2427251 2427250 2026-06-16T09:06:37Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2427251 wikitext text/x-wiki jboq3zhdrgwblsnofiyq051pdpaju55 2427252 2427251 2026-06-16T09:07:03Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.metamonitoring.downtime Downtime for 0:05:00 of prometheus/deadmanswitchnotified, prometheus/deadmanswitchonamdb, prometheus/extmon on 2 host(s) with reason: cookbook test 2427252 wikitext text/x-wiki 6t45pep48nsuexnu9bgunfe2clz5uj8 2427253 2427252 2026-06-16T09:07:05Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2427253 wikitext text/x-wiki 94wveq2pprbxgzq3f72id3qz9kmjh5b 2427254 2427253 2026-06-16T09:07:11Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.metamonitoring.downtime (exit_code=0) Downtime for 0:05:00 of prometheus/deadmanswitchnotified, prometheus/deadmanswitchonamdb, prometheus/extmon on 2 host(s) with reason: cookbook test 2427254 wikitext text/x-wiki 4cxg0dk1rmpyfwasd3d9i13brm5ncg7 2427255 2427254 2026-06-16T09:08:00Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2427255 wikitext text/x-wiki cd9e1tiedxke1yno4093s5uetl17jns 2427256 2427255 2026-06-16T09:09:09Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es1036.eqiad.wmnet with reason: host reimage 2427256 wikitext text/x-wiki klinnz79sdp1dxonk4fl5609q5ukup0 2427257 2427256 2026-06-16T09:13:57Z Stashbot 7414 urbanecm: php multiversion/MWScript.php WikimediaMaintenance:createExtensionTables.php --wiki={testwikidatawiki,wikidatawiki} growthexperiments # T418115, within mw-debug 2427257 wikitext text/x-wiki az83fxrowqb265huzjxc23zkwh9t7xn 2427258 2427257 2026-06-16T09:14:57Z Stashbot 7414 urbanecm@deploy1003: urbanecm: Continuing with deployment 2427258 wikitext text/x-wiki 80wkopkpxyuoh43i6o9w13xw9q5igkx 2427260 2427258 2026-06-16T09:18:23Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2427260 wikitext text/x-wiki swy7jtg0i1o4mgduz3mr5o4avyuqj40 2427261 2427260 2026-06-16T09:19:16Z Stashbot 7414 urbanecm@deploy1003: Finished scap sync-world: Backport for [[gerrit:1297161|[Growth] wikidatawiki: Enable Growth features (T418115)]] (duration: 16m 29s) 2427261 wikitext text/x-wiki rud0i4fkg9g0vy47e8s48f0dhjsen5n 2427262 2427261 2026-06-16T09:20:11Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2427262 wikitext text/x-wiki gvdrrt30jtirobxg9nlk4qn82wl2jlt 2427263 2427262 2026-06-16T09:21:51Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2427263 wikitext text/x-wiki 3la0zvrfzpvd1bwt7na2z580jragtnx 2427264 2427263 2026-06-16T09:23:37Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2427264 wikitext text/x-wiki 9i4dhwz98m3kpfnifi1g4t03d0ukhih 2427265 2427264 2026-06-16T09:24:45Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2427265 wikitext text/x-wiki 676iem1gstwo3v1sx40w0huo6ef9c67 2427266 2427265 2026-06-16T09:26:25Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2427266 wikitext text/x-wiki f72drufu48rtbdfq15032keu5hcilqu 2427267 2427266 2026-06-16T09:26:31Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es1036.eqiad.wmnet with OS trixie 2427267 wikitext text/x-wiki de1fivxhlsrlkxbv1c8wjhmr5m72kv0 2427268 2427267 2026-06-16T09:29:26Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 8 hosts with reason: Primary switchover es6 T429303 2427268 wikitext text/x-wiki ahly3fxo8jryzi80bypikd4nndxr1dt 2427269 2427268 2026-06-16T09:29:32Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2427269 wikitext text/x-wiki cwopa6valkawjf7aqqfa02ugcb94wh2 2427270 2427269 2026-06-16T09:29:40Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set es2037 with weight 0 T429303', diff saved to https://phabricator.wikimedia.org/P94162 and previous config saved to /var/cache/conftool/dbconfig/20260616-092937-marostegui.json 2427270 wikitext text/x-wiki 4qi4wdtmfp1s8o83v0h9px3u5t7uk3r 2427271 2427270 2026-06-16T09:30:04Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2427271 wikitext text/x-wiki 4udwto4u62yyc7g2boca8eyk15nck29 2427272 2427271 2026-06-16T09:30:06Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2427272 wikitext text/x-wiki 06jhnol70w88gurw7pdrjq7is71bdx9 2427274 2427272 2026-06-16T09:30:36Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2427274 wikitext text/x-wiki ndlawq4psvpca5gxcch0hlnt3peuodc 2427275 2427274 2026-06-16T09:30:53Z Stashbot 7414 marostegui: Starting es6 codfw failover from es2035 to es2037 - T429303 2427275 wikitext text/x-wiki kzu6b8eim6cmfcdq5dr02gq9bzegg99 2427276 2427275 2026-06-16T09:31:24Z Stashbot 7414 jayme: imported istioctl 1.29.4-1 to bookworm-/trixie-wikimedia - T427401 2427276 wikitext text/x-wiki dp2nbge6mbh52v8wbrohct9s6ivy9lg 2427277 2427276 2026-06-16T09:31:54Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote es2037 to es6 primary T429303', diff saved to https://phabricator.wikimedia.org/P94163 and previous config saved to /var/cache/conftool/dbconfig/20260616-093149-marostegui.json 2427277 wikitext text/x-wiki 8ld90h4g02x69103mjs0jeavmzccmc7 2427278 2427277 2026-06-16T09:32:50Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool es2035 T429303', diff saved to https://phabricator.wikimedia.org/P94164 and previous config saved to /var/cache/conftool/dbconfig/20260616-093247-marostegui.json 2427278 wikitext text/x-wiki ns3fi1dd5mf1khq5rcm8ab5w50lhlig 2427279 2427278 2026-06-16T09:34:16Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.major-upgrade 2427279 wikitext text/x-wiki kc8sbb6lge0oyu4qllne1n8mvvoq28b 2427280 2427279 2026-06-16T09:34:26Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool es2035: Upgrading es2035.codfw.wmnet 2427280 wikitext text/x-wiki 1nwu6i6m6km9j07iq2foj1ak1j8y2zq 2427281 2427280 2026-06-16T09:34:37Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool es2035: Upgrading es2035.codfw.wmnet 2427281 wikitext text/x-wiki j7nwyu5ztw7z0n6i1tob4ozb7ldfri1 2427283 2427281 2026-06-16T09:35:32Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host es2035.codfw.wmnet with OS trixie 2427283 wikitext text/x-wiki af5ma97k6ndwv1g7dh8t1hh9qu02k1d 2427284 2427283 2026-06-16T09:37:06Z Stashbot 7414 urbanecm@deploy1003: mwscript-k8s job started: extensions/GrowthExperiments/maintenance/refreshUserImpactData.php --registeredWithin=2week --hasEditsAtLeast=3 --ignoreIfUpdatedWithin=6hour --verbose --use-job-queue # T418115 2427284 wikitext text/x-wiki emdb9o4hdx89evigtdszpge2eki51x3 2427285 2427284 2026-06-16T09:37:14Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es1036: Migration of es1036.eqiad.wmnet completed 2427285 wikitext text/x-wiki qxnrfim0t1gual9pg67euc31w052qma 2427286 2427285 2026-06-16T09:37:40Z Stashbot 7414 urbanecm@deploy1003: mwscript-k8s job started: extensions/GrowthExperiments/maintenance/refreshUserImpactData.php --wiki=wikidatawiki --registeredWithin=1year --editedWithin=2week --hasEditsAtLeast=3 --ignoreIfUpdatedWithin=6hour --verbose --use-job-queue # T418115 2427286 wikitext text/x-wiki m9dqaiit8hflbh03elukl0bsuz48pbj 2427287 2427286 2026-06-16T09:37:43Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1302762|hCaptcha: Enable for UploadWizard on all wikis with it (T426126)]] 2427287 wikitext text/x-wiki jfaxzxytvjrhtt4ey5hsiuka88l6uuc 2427288 2427287 2026-06-16T09:38:01Z Stashbot 7414 urbanecm@deploy1003: mwscript-k8s job started: extensions/GrowthExperiments/maintenance/refreshUserImpactData.php --wiki=wikidatawiki --registeredWithin=2week --hasEditsAtLeast=3 --ignoreIfUpdatedWithin=6hour --verbose --use-job-queue # T418115 2427288 wikitext text/x-wiki 3poua632ba9mh412r50tir5qc3i90rh 2427289 2427288 2026-06-16T09:39:40Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1302762|hCaptcha: Enable for UploadWizard on all wikis with it (T426126)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2427289 wikitext text/x-wiki bm2bpmm77mekhe1y23qcrau4joa4exf 2427290 2427289 2026-06-16T09:43:00Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2427290 wikitext text/x-wiki lr9o2ol59zvylq57q3db1w8gcji4o61 2427291 2427290 2026-06-16T09:43:27Z Stashbot 7414 marostegui: Drop wrongly created table son testwikidatawiki s3 master T429304 2427291 wikitext text/x-wiki bn1tdcrtvynlfuht8l9zep1qgmsorlh 2427293 2427291 2026-06-16T09:47:19Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1302762|hCaptcha: Enable for UploadWizard on all wikis with it (T426126)]] (duration: 09m 38s) 2427293 wikitext text/x-wiki aemqfx4a21mxjuc93oz7xlhwl51wl2u 2427294 2427293 2026-06-16T09:48:59Z Stashbot 7414 urbanecm@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply 2427294 wikitext text/x-wiki 5e23wzgcoe3l6ts27wov7tzzg5ayrd2 2427295 2427294 2026-06-16T09:49:40Z Stashbot 7414 urbanecm@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply 2427295 wikitext text/x-wiki 2an9d0slw39suisif2hgvv5w8575h6v 2427297 2427295 2026-06-16T09:52:30Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on es2035.codfw.wmnet with reason: host reimage 2427297 wikitext text/x-wiki hfjz79owsqusbtl1g6jujim7nrgwxwv 2427298 2427297 2026-06-16T09:59:40Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2035.codfw.wmnet with reason: host reimage 2427298 wikitext text/x-wiki bn0mct3xmay1qokoegg81don2zi56uh 2427300 2427298 2026-06-16T10:17:10Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es2035.codfw.wmnet with OS trixie 2427300 wikitext text/x-wiki hmjiiykwzqau6qm5n0gbnel5050natu 2427301 2427300 2026-06-16T10:17:17Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply 2427301 wikitext text/x-wiki bx8dq11bthr4blyb31hz16r7tmefq1u 2427302 2427301 2026-06-16T10:18:16Z Stashbot 7414 kamila@deploy1003: helmfile [staging] START helmfile.d/services/shellbox: apply 2427302 wikitext text/x-wiki ixnd2km91w1fuz4ao3w8bi1hws6gpsv 2427303 2427302 2026-06-16T10:18:28Z Stashbot 7414 kamila@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox: apply 2427303 wikitext text/x-wiki 2ymkkippxe9d2cyly1hz4bwo9tqjxbw 2427304 2427303 2026-06-16T10:18:31Z Stashbot 7414 trueg@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply 2427304 wikitext text/x-wiki 12rx439szne2urchit8o3xmjc5wv0rc 2427305 2427304 2026-06-16T10:18:38Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox: apply 2427305 wikitext text/x-wiki b6lsmz3dcak9yqmysgkicuje1bqoien 2427306 2427305 2026-06-16T10:19:42Z Stashbot 7414 kamila@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox: apply 2427306 wikitext text/x-wiki 7av6udx0aezqfiysqj16udkcwa3v6gv 2427307 2427306 2026-06-16T10:20:44Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply 2427307 wikitext text/x-wiki ao62toe492e01knatpszgn2t7lutvlx 2427308 2427307 2026-06-16T10:20:56Z Stashbot 7414 jmm@dns1004: START - running authdns-update 2427308 wikitext text/x-wiki 2zbv3q1yj3ogak9875jthcloes7nowc 2427309 2427308 2026-06-16T10:21:09Z Stashbot 7414 jgiannelos@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply 2427309 wikitext text/x-wiki 3fw42guuoml81ifdpztjn1ttdf4wdol 2427310 2427309 2026-06-16T10:21:11Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] START helmfile.d/services/mw-parsoid: apply 2427310 wikitext text/x-wiki s4ojzhpnr9kdereq9nx52tjhx1mhzzm 2427311 2427310 2026-06-16T10:21:37Z Stashbot 7414 jgiannelos@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply 2427311 wikitext text/x-wiki p2ezvt4wz7mqw2po3y2mzxr1tx8pyjd 2427312 2427311 2026-06-16T10:22:09Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2427312 wikitext text/x-wiki c88l3s77d04oehnuu8jddfbj4yzg8rm 2427314 2427312 2026-06-16T10:22:37Z Stashbot 7414 jmm@dns1004: END - running authdns-update 2427314 wikitext text/x-wiki lr63td8soz5ruz11ndxwg0y9ckdgftp 2427315 2427314 2026-06-16T10:22:45Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es1036: Migration of es1036.eqiad.wmnet completed 2427315 wikitext text/x-wiki 2fcib47exk619jg46i4z2nnnani0t0c 2427316 2427315 2026-06-16T10:22:47Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2427316 wikitext text/x-wiki dzcyyf09o8a79drohqzkoz1la7tqh38 2427317 2427316 2026-06-16T10:24:07Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.remove-downtime for db1154.eqiad.wmnet 2427317 wikitext text/x-wiki 036197o9qv458wctjy2j4v8x9ktsqlp 2427318 2427317 2026-06-16T10:24:10Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for db1154.eqiad.wmnet 2427318 wikitext text/x-wiki c5v5rvali0642ca5fivaos7ak9si660 2427319 2427318 2026-06-16T10:24:12Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.remove-downtime for db1155.eqiad.wmnet 2427319 wikitext text/x-wiki qm91gztdfwy4hs6flxzzi9zaczzd3fh 2427320 2427319 2026-06-16T10:24:14Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for db1155.eqiad.wmnet 2427320 wikitext text/x-wiki clkm71mbjgo1302uv1xt87kqwk3k3as 2427321 2427320 2026-06-16T10:24:19Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.remove-downtime for 11 hosts 2427321 wikitext text/x-wiki fud2ijo59wqae7j3bxeoica9ftyzi5b 2427322 2427321 2026-06-16T10:24:25Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for 11 hosts 2427322 wikitext text/x-wiki oa3i6gexdw31kubhorqxqprzxllb9co 2427323 2427322 2026-06-16T10:24:41Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.remove-downtime for an-redacteddb1001.eqiad.wmnet 2427323 wikitext text/x-wiki r73rra8xdp36b2h6hds314rmux3l2u2 2427324 2427323 2026-06-16T10:24:43Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for an-redacteddb1001.eqiad.wmnet 2427324 wikitext text/x-wiki 5iictqru21wi7g4vqwxmawveiv0rl1g 2427325 2427324 2026-06-16T10:24:45Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool es2035: Migration of es2035.codfw.wmnet completed 2427325 wikitext text/x-wiki mj4vu2zimfb97m2kjv3tpvgqe5r07v4 2427326 2427325 2026-06-16T10:24:48Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox: apply 2427326 wikitext text/x-wiki mg9s4fva91u73z9hzqsd1tsy2vlxeul 2427327 2427326 2026-06-16T10:25:32Z Stashbot 7414 kamila@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox: apply 2427327 wikitext text/x-wiki h45a0bh8m2kl3g6o4nvxmdvm9est1i2 2427329 2427327 2026-06-16T10:49:34Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool es1037 T429118', diff saved to https://phabricator.wikimedia.org/P94172 and previous config saved to /var/cache/conftool/dbconfig/20260616-104931-marostegui.json 2427329 wikitext text/x-wiki 147ewcvs7x2lzys2upstlqoipype887 2427334 2427329 2026-06-16T11:06:33Z Stashbot 7414 moritzm: installing Bird security updates on routed Ganeti nodes 2427334 wikitext text/x-wiki 1cop2x1zfccsyli3r0qggxedvz8jdzd 2427335 2427334 2026-06-16T11:10:15Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool es2035: Migration of es2035.codfw.wmnet completed 2427335 wikitext text/x-wiki c8p70r7us7i9k67wuvj915iqfx0l7t6 2427336 2427335 2026-06-16T11:10:17Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) 2427336 wikitext text/x-wiki jiku3fr7p5h33uvigyy3gnhblmkguow 2427339 2427336 2026-06-16T11:42:00Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078 2427339 wikitext text/x-wiki a0bztt3ld3ui9tnxg5j0bq88ukml3ex 2427340 2427339 2026-06-16T11:42:18Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078 2427340 wikitext text/x-wiki b94q76vktdw84dgol9drsfhe3czmxbk 2427341 2427340 2026-06-16T11:43:05Z Stashbot 7414 dreamyjazz@deploy1003: Started scap sync-world: Backport for [[gerrit:1302794|Revert "hCaptcha: Enable for UploadWizard on all wikis with it"]] 2427341 wikitext text/x-wiki aiw4o1b5xvgu85t5og713114mp9dw44 2427342 2427341 2026-06-16T11:43:07Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.dns.netbox 2427342 wikitext text/x-wiki dpw3urwdjf605mx4szvj2wbznditees 2427343 2427342 2026-06-16T11:45:01Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Backport for [[gerrit:1302794|Revert "hCaptcha: Enable for UploadWizard on all wikis with it"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2427343 wikitext text/x-wiki dufxwetn9h6hn8q2l2td1ajzmb7mike 2427344 2427343 2026-06-16T11:46:05Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2427344 wikitext text/x-wiki b3ixks9gtl9jvaw41lazeap5lyzzk55 2427345 2427344 2026-06-16T11:46:16Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1078 2427345 wikitext text/x-wiki 5hpvamrt01przqrlv5t8pgia1plerzd 2427346 2427345 2026-06-16T11:46:36Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.mysql.depool depool db2153: codfw rack a5 depool for switch maintenance T428020 2427346 wikitext text/x-wiki 5cyq9qrkqub09806aq0a8n6zet78m6k 2427347 2427346 2026-06-16T11:46:57Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2153: codfw rack a5 depool for switch maintenance T428020 2427347 wikitext text/x-wiki nc1wx44b7w0ddc6h1ux5e7snahaeq90 2427348 2427347 2026-06-16T11:47:32Z Stashbot 7414 dreamyjazz@deploy1003: dreamyjazz: Continuing with deployment 2427348 wikitext text/x-wiki act6epqve9mdkwjx2sissawc6a07qke 2427349 2427348 2026-06-16T11:47:52Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.mysql.depool depool db2154: codfw rack a5 depool for switch maintenance T428020 2427349 wikitext text/x-wiki fu23bbfc5ju5i442906sa4tjeh4ai19 2427350 2427349 2026-06-16T11:48:22Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2154: codfw rack a5 depool for switch maintenance T428020 2427350 wikitext text/x-wiki jho3x7ukp2opy9gggmqqihklljkat5e 2427351 2427350 2026-06-16T11:48:25Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.mysql.depool depool db2157: codfw rack a5 depool for switch maintenance T428020 2427351 wikitext text/x-wiki tk9qq744jfgqve1tfpxz30q65aswg19 2427352 2427351 2026-06-16T11:48:41Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1078 2427352 wikitext text/x-wiki d3lsagtj0rgqcg7om0xaxge8lj22zn3 2427353 2427352 2026-06-16T11:48:46Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2157: codfw rack a5 depool for switch maintenance T428020 2427353 wikitext text/x-wiki hk4siz7iwnlshx71w8lc45c68n7g9rr 2427354 2427353 2026-06-16T11:48:50Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.mysql.depool depool db2175: codfw rack a5 depool for switch maintenance T428020 2427354 wikitext text/x-wiki maybi19zrljzlj3x667g8t4dy7sm71i 2427356 2427354 2026-06-16T11:49:10Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2175: codfw rack a5 depool for switch maintenance T428020 2427356 wikitext text/x-wiki engiubdjtxo02ldiaq3xl1ux9naikhg 2427357 2427356 2026-06-16T11:49:14Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.mysql.depool depool db2176: codfw rack a5 depool for switch maintenance T428020 2427357 wikitext text/x-wiki 4v9abnlyyxi2uqszzoltbxv5sihtqj8 2427358 2427357 2026-06-16T11:49:34Z Stashbot 7414 cmooney@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2176: codfw rack a5 depool for switch maintenance T428020 2427358 wikitext text/x-wiki qcd61s3o3rmhp0th25lfgo0kln12ebe 2427359 2427358 2026-06-16T11:51:50Z Stashbot 7414 dreamyjazz@deploy1003: Finished scap sync-world: Backport for [[gerrit:1302794|Revert "hCaptcha: Enable for UploadWizard on all wikis with it"]] (duration: 08m 45s) 2427359 wikitext text/x-wiki reeg1oryk0i7oqjflvi9nyhbkn4hqyp 2427360 2427359 2026-06-16T11:57:04Z Stashbot 7414 cmooney@cumin1003: START - Cookbook sre.k8s.pool-depool-node depool for host ml-serve2001.codfw.wmnet 2427360 wikitext text/x-wiki 1uzdmxfzk2pf9sdhx60n2tfu58029eh 0 11689 2427232 2423298 2026-06-16T08:37:00Z BRouberol-WMF 39141 2427232 wikitext text/x-wiki r5xhcbyxwga63jbojiojvcxdfbiypui 0 17290 2427105 2426601 2026-06-15T18:26:49Z Stashbot 7414 dancy: Rebooting deployment-jobrunner05 2427105 wikitext text/x-wiki g5szubth3pa5e1g6xef2uu31hqhk44u 2427163 2427105 2026-06-15T20:26:02Z Stashbot 7414 dancy: Added deployment-db16 (T429245) 2427163 wikitext text/x-wiki hkj9wuz4weig8f6fj5jo9oaclmkqodj 2427165 2427163 2026-06-15T20:48:51Z Stashbot 7414 dancy: deployment-db11.deployment-prep will be going read-only soon while db15 is being promoted to primary. 2427165 wikitext text/x-wiki owq6igxbejqup9smkqc8noqjc3ho48x 2427167 2427165 2026-06-15T20:53:30Z Stashbot 7414 dancy: deployment-db11.deployment-prep going read-only 2427167 wikitext text/x-wiki 98wki7yvxtq45s25rw7qp7wc1ef0a9g 2427168 2427167 2026-06-15T21:00:51Z Stashbot 7414 dancy: deployment-db15 promoted to master (T428930) 2427168 wikitext text/x-wiki 6tj5uotxm16tz08m53ups1bav8xhw6q 2427192 2427168 2026-06-15T23:40:07Z Stashbot 7414 dancy: systemctl restart php8.3-fpm on deployment-jobrunner05 to reload beta db configuration (T428930) 2427192 wikitext text/x-wiki 9qiduedcocrwk45byx6l58bstyne9mc 2 21112 2427155 2426654 2026-06-15T20:01:57Z JJMC89 7474 JJMC89 moved page [[Joal]] to [[User:Joal~labswiki]] over a redirect without leaving a redirect: Vandalism 1961894 wikitext text/x-wiki dv993cexn2y1ir9s6zithgx2obk1l9f 0 22503 2427196 2422034 2026-06-16T01:17:24Z Neil Shah-Quinn (WMF) 36601 Clarify how to start using a wprov value. Extensively reorganize and copyedit. 2427196 wikitext text/x-wiki elo10eakooveotiflmtszplav9wk9js 498 30942 2426903 2426553 2026-06-15T12:46:44Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.upgrade_mons (T428385) 2426903 wikitext text/x-wiki f4fmfc42zlevcpbts9pjdmkvhr3g0w4 2426914 2426903 2026-06-15T13:08:27Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.upgrade_mons (exit_code=99) 2426914 wikitext text/x-wiki 5s0fllqr6ijkj2kz8bg6ynob9s9dv1c 2426947 2426914 2026-06-15T13:38:41Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.upgrade_osds (T428385) 2426947 wikitext text/x-wiki oaj2jrydtvgchytm9bs0t8dcz7dib0t 2426953 2426947 2026-06-15T13:48:29Z Stashbot 7414 andrew@cloudcumin1001: END (ERROR) - Cookbook wmcs.ceph.upgrade_osds (exit_code=97) 2426953 wikitext text/x-wiki jrb6aiek1tstc5e9mi7q2hr47tolcd9 2426961 2426953 2026-06-15T13:50:38Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.upgrade_mons (T428385) 2426961 wikitext text/x-wiki 45zfyxvori1weae6tyomrvyblgr1oo7 2427000 2426961 2026-06-15T14:10:15Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.upgrade_mons (exit_code=0) 2427000 wikitext text/x-wiki ep1fhc4k7ny7roz8z7i1pfja9oe4yid 2427015 2427000 2026-06-15T14:32:01Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.upgrade_osds (T428385) 2427015 wikitext text/x-wiki nudjmp67v175w74hbtyvnrbc9h6591o 2427080 2427015 2026-06-15T17:07:33Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.upgrade_osds (exit_code=99) 2427080 wikitext text/x-wiki ow5cclle5z27qgucf0f5gkqrw2khd0x 2427106 2427080 2026-06-15T18:27:04Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.upgrade_osds (T428385) 2427106 wikitext text/x-wiki 520xpql63u2ujc60y8jta604sq1phwr 2427111 2427106 2026-06-15T18:40:28Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.upgrade_osds (exit_code=99) 2427111 wikitext text/x-wiki qyf7265x6mz5qxwepf0g54njp1ttgvd 2427118 2427111 2026-06-15T19:00:13Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.upgrade_osds (T428385) 2427118 wikitext text/x-wiki 3gh7oubbcu07iublmcjygf6rcx5sttg 2427134 2427118 2026-06-15T19:32:45Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.upgrade_osds (exit_code=0) 2427134 wikitext text/x-wiki agxrfbotktm0stc0xxt2tjrt1q7ddmt 2427152 2427134 2026-06-15T19:46:03Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.upgrade_osds (T428385) 2427152 wikitext text/x-wiki jvjeyr46f1u6mritf2tnc51qqvtc3en 2427169 2427152 2026-06-15T21:03:00Z Stashbot 7414 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.ceph.upgrade_osds (exit_code=99) 2427169 wikitext text/x-wiki 7ctyry0whjapw9l6mc0jhee3vbnu6ni 2427170 2427169 2026-06-15T21:04:25Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.upgrade_osds (T428385) 2427170 wikitext text/x-wiki t1nhoeyf6sx0he2r5zzj6f9xlnz8bbl 2427180 2427170 2026-06-15T21:48:54Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.upgrade_osds (exit_code=0) 2427180 wikitext text/x-wiki pbhjsx0g6tq897wlrv5ohniqecai4kz 0 442261 2427073 2308175 2026-06-15T16:34:07Z Atsuko-WMF 54082 Moved from CirrusSearch to OpenSearch-on-K8S 2427073 wikitext text/x-wiki 78tgkr0f843zgxwkwra2w3bwqif35z4 498 443628 2427019 2426623 2026-06-15T14:45:08Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27554179755 (https://github.com/cluebotng/component-configs/commits/a236330774424b9ce999258a01f924f1994594b1) 2427019 wikitext text/x-wiki q16sgw9u972vjcevywv8js20ax2bm37 0 444245 2427355 2416929 2026-06-16T11:48:53Z CMooney (WMF) 24288 /* Known issues */ 2427355 wikitext text/x-wiki h0vkq6qr8olfjl73aju93m0z7jcweu8 0 446656 2426930 2401108 2026-06-15T13:21:56Z Tiziano Fogli 37623 /* The IRC bot has quit my channel and has not joined yet, what is up? */ 2426930 wikitext text/x-wiki grxc94r642zcz23b47aabosj5kk49q2 0 447310 2427002 2426302 2026-06-15T14:14:22Z Phuedx (WMF) 47850 /* Test Kitchen maintainer documentation */ 2427002 wikitext text/x-wiki 1uza9o6v52w9aw32mp3aptk9wdi66nn 0 447862 2426954 2425446 2026-06-15T13:48:34Z FCeratto-WMF 39104 Upgrading sanitarium masters 2426954 wikitext text/x-wiki o4dinyvyqenj8tasmfj9upfjcoraa2p 2426990 2426954 2026-06-15T14:05:10Z FCeratto-WMF 39104 Update sanitarium 2426990 wikitext text/x-wiki gcjs7mxjhv3f8f2xb9iwurg13na44uh 2427001 2426990 2026-06-15T14:10:21Z FCeratto-WMF 39104 Upgrading sanitarium masters 2427001 wikitext text/x-wiki kxv75fwx31m7xq95h7a99qjmg584r2q 0 449160 2427193 2426716 2026-06-16T00:01:55Z Dexbot 30554 Bot: Updating the report 2427193 wikitext text/x-wiki nurf2gczutsa76cnlwclb7yax7xxy86 2427273 2427193 2026-06-16T09:30:11Z Dexbot 30554 Bot: Updating the report 2427273 wikitext text/x-wiki 34cuvol1zql4hko7jkm88ih3rd4mbr3 2427331 2427273 2026-06-16T10:51:24Z Dexbot 30554 Bot: Updating the report 2427331 wikitext text/x-wiki dosdhnahb0md4oh6n87ptis4hljr2ei 0 449339 2427259 2078302 2026-06-16T09:15:37Z JMeybohm (WMF) 16709 Add packaging instructions for istioctl 2427259 wikitext text/x-wiki bmzry8h7fa1i6l89vels4md5syi0xs0 498 451074 2427086 2379448 2026-06-15T17:20:01Z Stashbot 7414 volans@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase by 24 cores, 20 gigabytes, 32768 ram (T427731) 2427086 wikitext text/x-wiki ldzjqcxfu0a4vfs0y3ovuev2me1613n 2427087 2427086 2026-06-15T17:20:08Z Stashbot 7414 volans@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) by 24 cores, 20 gigabytes, 32768 ram (T427731) 2427087 wikitext text/x-wiki 0vxyryznp0u6m4bk1jkpxmrj3x0adtz 0 451075 2427164 2423116 2026-06-15T20:29:36Z DAdedoyin (WMF) 39150 Chaos crew rotation 2427164 wikitext text/x-wiki b0r37jaeme6frusl2w3mmlapqqw6lto 498 452890 2427018 2426649 2026-06-15T14:44:32Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27554179713 (https://github.com/cluebotng/component-configs/commits/a236330774424b9ce999258a01f924f1994594b1) 2427018 wikitext text/x-wiki pc9fklowap4gewoorwpuvj0l753qk1d 2427079 2427018 2026-06-15T17:03:58Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27562697218 (https://github.com/cluebotng/component-configs/commits/5c1faa1ebe1b269c9d3e13fc4d7c7a9fddf23bf2) 2427079 wikitext text/x-wiki 2iw3rfixq3fdg6ow3e78ypadiy6j29b 2427084 2427079 2026-06-15T17:11:24Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27562889711 (https://github.com/cluebotng/component-configs/commits/b9c48365007a90ed1c240655306615542751ea6f) 2427084 wikitext text/x-wiki 0c5wjbv0aorm0ljdpjd1jkxt1xzywqx 116 453906 2426992 2426813 2026-06-15T14:06:26Z Gitlabaccountapprovalbot 37332 ayaan was rejected. 2426992 wikitext text/x-wiki 92wz8oa53x9hb23az57loio32yb7r9c 2427010 2426992 2026-06-15T14:24:20Z Gitlabaccountapprovalbot 37332 enlisar was rejected. 2427010 wikitext text/x-wiki jgtxgoyi8rpipedr49fom0xcjnwyxam 2427191 2427010 2026-06-15T22:39:24Z Gitlabaccountapprovalbot 37332 @mohammadhijjawi was approved. 2427191 wikitext text/x-wiki jc9ykrr6gtd7mw2zrg5r8wq06t9uums 498 454177 2427113 2210834 2026-06-15T18:44:00Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: toolforge jobs load jobs.yaml # update jobs.yaml and redeploy jobs to silence “the running version of job 'logrotate/monitor' is different from what was configured, please recreate or redeploy” warnings 2427113 wikitext text/x-wiki 9le4fbm0z0ohy8o92lpzm7y415hy02w 0 455928 2427296 2419915 2026-06-16T09:51:00Z Phuedx (WMF) 47850 Remove drifted API documentation. Add link to documentation microsite. 2427296 wikitext text/x-wiki fgmfbcqtjdvvpeg1stuibdnkgk6s5i0 0 456086 2427328 2408071 2026-06-16T10:48:50Z FCeratto-WMF 39104 Add dev hints 2427328 wikitext text/x-wiki 2xmxmlbvijfcy1rl9nhuzoogtf96kdf 2427330 2427328 2026-06-16T10:50:09Z FCeratto-WMF 39104 Minor fix 2427330 wikitext text/x-wiki 45bx10iud37h0nhujmvir3e7chshgpf 498 458787 2427017 2426624 2026-06-15T14:44:31Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27554179699 (https://github.com/cluebotng/component-configs/commits/a236330774424b9ce999258a01f924f1994594b1) 2427017 wikitext text/x-wiki jmis3lc12qrtzvogpa02ppfd2fts1y0 2427362 2427017 2026-06-16T11:58:40Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27615770915 (https://github.com/cluebotng/component-configs/commits/38c93594afd6d57d84fde7b3d927c1c73e0a18f9) 2427362 wikitext text/x-wiki hqtcqe81egxf2h08uk9jqn7gqwtcwsq 0 459189 2427197 2339885 2026-06-16T01:20:00Z Neil Shah-Quinn (WMF) 36601 Fix typo in date 2427197 wikitext text/x-wiki d84y69vtdrmxvua4zqv9tbdw0540lha 498 459243 2427020 2425661 2026-06-15T14:52:52Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27554700598 (https://github.com/cluebotng/component-configs/commits/12298f8c7711b0dbc3ebe3196da055b62b307301) 2427020 wikitext text/x-wiki n0a8uli231drgn09zc72qbj2shct7bo 2427045 2427020 2026-06-15T15:26:32Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27556781475 (https://github.com/cluebotng/component-configs/commits/81d6af2f5fa912449070fe6ac104761023a9960b) 2427045 wikitext text/x-wiki gc3j3wt3m0frx223ldmmtw9fgoe9ehx 2427047 2427045 2026-06-15T15:31:05Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27557053203 (https://github.com/cluebotng/component-configs/commits/80c3b871693d13296e1c4640105e4a8cabc9d5a4) 2427047 wikitext text/x-wiki ewx5c9bcmf05svsk5zq9odcb7li4jz0 2427072 2427047 2026-06-15T16:24:31Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/27560286658 (https://github.com/cluebotng/component-configs/commits/5c1faa1ebe1b269c9d3e13fc4d7c7a9fddf23bf2) 2427072 wikitext text/x-wiki ls81rn0nfypyevlsnngfg5b2rs6e1t1 2427361 2427072 2026-06-16T11:57:50Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/27615771202 (https://github.com/cluebotng/component-configs/commits/38c93594afd6d57d84fde7b3d927c1c73e0a18f9) 2427361 wikitext text/x-wiki sr26nve9ay4ycg6xfrkpvu0yly4lt9a 0 459290 2427032 2426805 2026-06-15T15:08:13Z JAllemandou (WMF) 37709 /* YYYY-MM-DD NEXT TUESDAY TRAIN (REPLACE THIS AFTER DEPLOY) */ Add a pgeview aloowlist change 2427032 wikitext text/x-wiki cwv5qy0mxn7oj7617ou3iodhbuf2zwa 0 459695 2427116 2426447 2026-06-15T18:46:06Z APaskulin (WMF) 14880 Update wiki status 2427116 wikitext text/x-wiki t19dtinqilt7oixgzxlezzkldjpol63 0 459699 2427194 2426840 2026-06-16T01:12:51Z Stashbot 7414 eileen: updated SmashPig revision e82c2c5f -> 8ad963c7 2427194 wikitext text/x-wiki 8ds28lqskr7lxnv4irxdlezv4q92azi 2427195 2427194 2026-06-16T01:16:31Z Stashbot 7414 eileen: config revision changed from 7ca9f992 to 8334c030 2427195 wikitext text/x-wiki cti7mxlnr2u5tfbjtme63r7xdci2ujh 2427198 2427195 2026-06-16T03:02:34Z Stashbot 7414 eileen: SmashPig upgraded from 8ad963c7 to 611947be 2427198 wikitext text/x-wiki a0oqnyrobxoktviji03p28ti61cxdjw 0 459719 2426998 2414860 2026-06-15T14:09:12Z BTullis (WMF) 25295 /* Service Level Objectives */ 2426998 wikitext text/x-wiki iyt1hbniwurczp8wbu9i8s6hcjvnrl9 2426999 2426998 2026-06-15T14:09:23Z BTullis (WMF) 25295 /* Reconciliation */ 2426999 wikitext text/x-wiki f1ytpco6xvxlrd1qzt21gvj5lrpfqgo 2427004 2426999 2026-06-15T14:18:44Z BTullis (WMF) 25295 /* Service Level Indicators (SLIs) */ 2427004 wikitext text/x-wiki 6spalymcyee9cnvui6ynn3pbyu2t08f 2427012 2427004 2026-06-15T14:26:36Z BTullis (WMF) 25295 2427012 wikitext text/x-wiki p78b9jhuyc7o4gplkcudc4ijwlobzdc 2427044 2427012 2026-06-15T15:25:30Z GLederrey (WMF) 6860 /* Service Level Objectives */ 2427044 wikitext text/x-wiki of9ud7oi9cb2in9m1m2b0b3pmfqm3cw 0 459783 2427023 2423050 2026-06-15T14:54:23Z Duesentrieb 52342 /* Architecture */ 2427023 wikitext text/x-wiki oaf0ivj0xh2nkksz2ivqvxg3jw7vv0l 0 459922 2427154 2416308 2026-06-15T19:47:28Z JKilani-WMF 37651 Removing application software to gitlab rather 2427154 wikitext text/x-wiki ch2c2f7rn7q3d9v2anvzdmayk0vf1py 0 460161 2427153 2421285 2026-06-15T19:46:46Z ArielGlenn 11 fix typo 2427153 wikitext text/x-wiki jqa7mrw9brc7bzi9gc2k2yxhhjugt0v 0 460233 2427124 2424464 2026-06-15T19:08:47Z MPopov (WMF) 5891 /* Creating a new fact table */ Boolean columns 2427124 wikitext text/x-wiki tkhfosbw2eq92xal0wdpajyf80yxbia 0 460331 2426922 2426762 2026-06-15T13:14:55Z Matma Rex 562 /* Developing JS code running on auth.wikimedia.org */ Typo 2426922 wikitext text/x-wiki oiozs89noccez7d5tq77l7wp8dsj2rb 2426925 2426922 2026-06-15T13:18:31Z Matma Rex 562 2426925 wikitext text/x-wiki k7bhh4o4vz0p98pmultafeozegj0s9l 2426926 2426925 2026-06-15T13:18:44Z Matma Rex 562 Undid revision [[Special:Diff/2426925|2426925]] by [[Special:Contributions/Matma Rex|Matma Rex]] ([[User talk:Matma Rex|talk]]) 2426926 wikitext text/x-wiki oiozs89noccez7d5tq77l7wp8dsj2rb 2426987 2426926 2026-06-15T14:04:30Z ESanders (WMF) 953 /* Developing JS code running on auth.wikimedia.org */ fix unbalanced tags 2426987 wikitext text/x-wiki d1wqpo018yo1v4lx8vnt3j3gra7tcvd 2 460338 2426957 2026-06-15T13:49:43Z ESanders (WMF) 953 test 2426957 wikitext text/x-wiki aj69rlymkxa12g3b3v5qgcjmpnztxdv 2426959 2426957 2026-06-15T13:50:07Z ESanders (WMF) 953 2426959 wikitext text/x-wiki 46rjuewvimoixn2ytzobj3dypxkj3m1 2426960 2426959 2026-06-15T13:50:38Z ESanders (WMF) 953 2426960 wikitext text/x-wiki o3bus8mbevatxzqur920z0zdgdgcxg3 2426966 2426960 2026-06-15T13:54:52Z ESanders (WMF) 953 2426966 wikitext text/x-wiki o6erdyfyeltzt269h7d5392a2upv494 2426970 2426966 2026-06-15T13:57:42Z ESanders (WMF) 953 2426970 wikitext text/x-wiki oiozs89noccez7d5tq77l7wp8dsj2rb 2426971 2426970 2026-06-15T13:57:59Z ESanders (WMF) 953 2426971 wikitext text/x-wiki ilyw0ho2ei2zhwy1sd7qv2iopt4xxtq 2426972 2426971 2026-06-15T13:58:20Z ESanders (WMF) 953 2426972 wikitext text/x-wiki f1jy5lq26y3p5q1ms2cja5fkabv7twm 2426973 2426972 2026-06-15T13:58:40Z ESanders (WMF) 953 2426973 wikitext text/x-wiki s1cso0matryfvd2kba6qkmobk2nhwun 2426974 2426973 2026-06-15T13:58:55Z ESanders (WMF) 953 2426974 wikitext text/x-wiki 4sst9twshb6g4s3esektpwcnaugtseu 2426975 2426974 2026-06-15T13:59:12Z ESanders (WMF) 953 2426975 wikitext text/x-wiki cj7jgwnj85vtudne60qzrrxksrnzoig 2426977 2426975 2026-06-15T14:00:31Z ESanders (WMF) 953 2426977 wikitext text/x-wiki oa6y6438xzbcr5eqcs8tjcdrkq8cxdc 2426979 2426977 2026-06-15T14:00:50Z ESanders (WMF) 953 2426979 wikitext text/x-wiki mlu2hyxnjyv3wy052ml8wh2goax2b5i 2426981 2426979 2026-06-15T14:01:37Z ESanders (WMF) 953 2426981 wikitext text/x-wiki m1hpl11uard8qntqfuj9k6rviulicm4 2427007 2426981 2026-06-15T14:22:53Z ESanders (WMF) 953 2427007 wikitext text/x-wiki 48aamobd1ngutuz7dcs8d49txitgfyt 498 460339 2427220 2026-06-16T07:54:25Z Labslogbot 55 Auto update of instance info. 2427220 wikitext text/x-wiki ls8l3j8xvednz5jtcyfrcm1mfvwrnqk 2 460340 2427313 2026-06-16T10:22:14Z Phuedx (WMF) 47850 Initial import for drafting 2427313 wikitext text/x-wiki 84teq2ijp8qsuyl7qfvge33cracdj20