Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.16 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 12 558 2449162 2390471 2026-08-18T14:28:35Z Personal computer loei 58718 2449162 wikitext text/x-wiki q6olwwv17didcsnonff7hir3j26kmq2 2449464 2449162 2026-08-19T07:31:14Z JJMC89 7474 Reverted edit by [[Special:Contributions/Personal computer loei|Personal computer loei]] ([[User talk:Personal computer loei|talk]]) to last revision by [[User:Yug|Yug]] 2390471 wikitext text/x-wiki c1ws2ccwbqpkmpyzv6dxct7c5hurxgm 498 3086 2449106 2448892 2026-08-18T13:28:33Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2449106 wikitext text/x-wiki eqzn70wgjo5eibz7oo7ktyc1sal66d7 2449110 2449106 2026-08-18T13:42:20Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2449110 wikitext text/x-wiki b7md01qiwbw84c6gg67b0fl57isa5xf 2449166 2449110 2026-08-18T14:35:50Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway 2449166 wikitext text/x-wiki b0e5i8n2nqp2yvejnk4961wtgz42jg2 2449167 2449166 2026-08-18T14:40:04Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway 2449167 wikitext text/x-wiki 1md9dm1dbo9cou4i8f1gx8pi3kav1m1 2449233 2449167 2026-08-18T16:46:34Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logging 2449233 wikitext text/x-wiki juqfak37lo9cgco8nr8bfxv3n58j9rd 2449235 2449233 2026-08-18T16:55:27Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logging 2449235 wikitext text/x-wiki 75ddfmubobyucyzzp0jkvu9f9537quy 2449236 2449235 2026-08-18T17:00:06Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2449236 wikitext text/x-wiki 6ye65xzuukt4ozo6r39dz4ptbweldow 2449243 2449236 2026-08-18T17:10:33Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api 2449243 wikitext text/x-wiki e7a77lr9x92iayxywfouwg2q9koh1j0 2449600 2449243 2026-08-19T10:37:38Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-cli 2449600 wikitext text/x-wiki apgbjnkuzkha1z25xq0lktub8kc3t5d 2449601 2449600 2026-08-19T10:40:54Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-cli 2449601 wikitext text/x-wiki jxjr7n8ru231ggh5scdniukaqxxirva 0 3863 2449277 2376215 2026-08-18T18:04:59Z AKlapper (WMF) 421 remove ancient reference to "current version" 2449277 wikitext text/x-wiki t59zhelgbiw2i2divc41q1t6opn839b 0 4108 2449117 2448789 2026-08-18T13:44:41Z ScheduleDeploymentBot 37566 Add [[gerrit:1326838]] to Tuesday, August 18 UTC afternoon backport window 2449117 wikitext text/x-wiki 21ffay6uabc8zsr6zn1ky1vrpdvakjd 2449190 2449117 2026-08-18T15:21:45Z ScheduleDeploymentBot 37566 Add [[gerrit:1326848]] to Tuesday, August 18 UTC late backport window 2449190 wikitext text/x-wiki pts5uxe2v8esjhgzjblj04u7vzqt0we 2449191 2449190 2026-08-18T15:26:28Z ADancy (WMF) 17694 /* {{Deployment_day|date=2026-08-18}} */ Add https://gerrit.wikimedia.org/r/c/operations/puppet/+/1324278 to puppet window 2449191 wikitext text/x-wiki 38me6ckmb8sf5auiyhg6ibj5k3j39ye 2449201 2449191 2026-08-18T15:54:04Z CMing (WMF) 23277 /* {{Deployment_day|date=2026-08-18}} */ 2449201 wikitext text/x-wiki 56ta45arb2c8a6xbftq1i7ck0hbibeh 2449212 2449201 2026-08-18T16:07:40Z ScheduleDeploymentBot 37566 Add [[gerrit:1326870]] to Tuesday, August 18 UTC late backport window 2449212 wikitext text/x-wiki i5eprr7sq06at8xgigey1tgm1zkvmqb 2449213 2449212 2026-08-18T16:07:46Z ScheduleDeploymentBot 37566 Add [[gerrit:1326871]] to Tuesday, August 18 UTC late backport window 2449213 wikitext text/x-wiki b76sgi2bkmuzmqky8cjm7axhvhxajqf 2449280 2449213 2026-08-18T18:10:21Z ScheduleDeploymentBot 37566 Add [[gerrit:1326896]] to Tuesday, August 18 UTC late backport window 2449280 wikitext text/x-wiki 5vdhdana0qwqlsm5sds9arbigw77ipb 2449281 2449280 2026-08-18T18:10:28Z ScheduleDeploymentBot 37566 Add [[gerrit:1326897]] to Tuesday, August 18 UTC late backport window 2449281 wikitext text/x-wiki hwt0vnmh4e30b7rkh8xxkhnlj6l0zkm 2449344 2449281 2026-08-18T20:41:51Z ScheduleDeploymentBot 37566 Add [[gerrit:1326925]] to Tuesday, August 18 UTC late backport window 2449344 wikitext text/x-wiki cz1dza25mqkpz18hl8orqr1xbc2v1qy 2449345 2449344 2026-08-18T20:42:10Z ScheduleDeploymentBot 37566 Add [[gerrit:1326926]] to Tuesday, August 18 UTC late backport window 2449345 wikitext text/x-wiki rns97m27byfyhfrnqw9qo2fy55kojc1 2449424 2449345 2026-08-19T04:33:32Z ScheduleDeploymentBot 37566 Add [[gerrit:1325532]] to Wednesday, August 19 UTC afternoon backport window 2449424 wikitext text/x-wiki cg8zo12vhbgpgyf5u5awy4c0qz1c8ou 2449645 2449424 2026-08-19T11:50:33Z ScheduleDeploymentBot 37566 Add [[gerrit:1327094]] to Wednesday, August 19 UTC afternoon backport window 2449645 wikitext text/x-wiki jxi8ak8vqy7tho6avda9vvjml3gyebt 0 4235 2449597 2448315 2026-08-19T10:33:10Z BTullis (WMF) 25295 /* Verifying before applying */ Added an example image 2449597 wikitext text/x-wiki pdjqy1v9vus4iz4ql7ap9jakn4l89ul 2449599 2449597 2026-08-19T10:36:13Z BTullis (WMF) 25295 /* Preserving Data */ 2449599 wikitext text/x-wiki s6qlc8udc8ly3mzjetn6qfnw9vfkl45 498 6726 2449093 2448869 2026-08-18T13:11:31Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2449093 wikitext text/x-wiki sqf1x58yu89ia81eomqarx3y1aljbnd 2449094 2449093 2026-08-18T13:11:35Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api 2449094 wikitext text/x-wiki ivdlehgtmtd2x4hbr3e2o0wli71fghs 2449097 2449094 2026-08-18T13:14:24Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2449097 wikitext text/x-wiki 3qhqpgux05wpgxlohv09pm791mlvb4k 2449105 2449097 2026-08-18T13:28:15Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2449105 wikitext text/x-wiki s2jofvhd8mu1fxd5f8d1u9spkv2cmhq 2449163 2449105 2026-08-18T14:31:00Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway 2449163 wikitext text/x-wiki c9ha6hm9ytamfmqdca39fnhdqukiqq5 2449165 2449163 2026-08-18T14:34:38Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway 2449165 wikitext text/x-wiki qxr9hmu3weunp9q7whgepyyaf0rg45j 2449204 2449165 2026-08-18T15:57:42Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logs-api 2449204 wikitext text/x-wiki p6lc6wi29t44l3ogl7awvwqzjknn630 2449211 2449204 2026-08-18T16:07:15Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logs-api 2449211 wikitext text/x-wiki tko53z35x62utm60cjg7wlaeq1y9yhj 2449221 2449211 2026-08-18T16:33:05Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logging 2449221 wikitext text/x-wiki qcye02qwgtf3d1vy4stdy0swknhg4ft 2449222 2449221 2026-08-18T16:33:35Z Stashbot 7414 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logging 2449222 wikitext text/x-wiki mk8yj200zf5wj3v0s61b84yqr46qv1b 2449224 2449222 2026-08-18T16:35:45Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logging 2449224 wikitext text/x-wiki evz7diqqm75lvufjhq8thzz1hka895n 2449232 2449224 2026-08-18T16:44:14Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logging 2449232 wikitext text/x-wiki s1yai4x8edmq2q4kgyzamkq668gudem 2449596 2449232 2026-08-19T10:30:46Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-cli 2449596 wikitext text/x-wiki mmxc29akvzdu1u46e4r5qvhmvyazmpv 2449598 2449596 2026-08-19T10:33:57Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-cli 2449598 wikitext text/x-wiki sm7enby6urigublpe8aqugd9ysvu43y 0 7919 2449015 2449014 2026-08-18T12:02:00Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1128-1134,1142-1148].eqiad.wmnet 2449015 wikitext text/x-wiki dzwoo43tjlzv4rs67q4halna9vaf3v3 2449016 2449015 2026-08-18T12:02:03Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2046.codfw.wmnet 2449016 wikitext text/x-wiki dcgq3dnoinywyt5m48bdm91nz5jjgnd 2449017 2449016 2026-08-18T12:02:09Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1128-1134,1142-1148].eqiad.wmnet 2449017 wikitext text/x-wiki i36gw7bo1dfq9x05gcqnv7vt7t3xal5 2449018 2449017 2026-08-18T12:02:40Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1149-1153,1158,1240-1247].eqiad.wmnet 2449018 wikitext text/x-wiki ckuostg6zfyj7wozu3owwc7vmm3xlas 2449019 2449018 2026-08-18T12:05:16Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2026.codfw.wmnet 2449019 wikitext text/x-wiki 0hco666p2klwx4njs66482oxud432yn 2449020 2449019 2026-08-18T12:06:52Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.kafka.roll-restart-reboot-brokers (exit_code=0) rolling reboot on A:kafka-test-eqiad 2449020 wikitext text/x-wiki nkvhxzghdh2kpf4w7xo4bco6qjkkana 2449021 2449020 2026-08-18T12:10:30Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1168: Pool db1168.eqiad.wmnet in after cloning 2449021 wikitext text/x-wiki 50m84zj5z4xbg5zsdnd4vgf3vzt8rc8 2449022 2449021 2026-08-18T12:10:38Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1168.eqiad.wmnet onto db1282.eqiad.wmnet 2449022 wikitext text/x-wiki t6rshcld9jvf6oid7n2l4lilxfu6w4p 2449023 2449022 2026-08-18T12:11:13Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1149-1153,1158,1240-1247].eqiad.wmnet 2449023 wikitext text/x-wiki f4yljksgt92ta2fb0g5wasz7pffoyjc 2449025 2449023 2026-08-18T12:19:20Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1149-1153,1158,1240-1247].eqiad.wmnet 2449025 wikitext text/x-wiki ljyokrkx8m9lisb8ik1q7fz0o9cijdp 2449026 2449025 2026-08-18T12:19:28Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1149-1153,1158,1240-1247].eqiad.wmnet 2449026 wikitext text/x-wiki 30ca06b9jesr5gzaqbtcp2x62266r9i 2449027 2449026 2026-08-18T12:19:58Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1248-1261].eqiad.wmnet 2449027 wikitext text/x-wiki csu4ptmy0r7yu7yb7h0h3dswlw5hrt7 2449028 2449027 2026-08-18T12:21:09Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1169: Pool db1169.eqiad.wmnet in after cloning 2449028 wikitext text/x-wiki b9b2p3ss75qts9vdovbklbuio8y0x37 2449029 2449028 2026-08-18T12:21:17Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1169.eqiad.wmnet onto db1283.eqiad.wmnet 2449029 wikitext text/x-wiki hbhk72ypu1dk0z6a8t0cleywuyc0kj4 2449030 2449029 2026-08-18T12:21:40Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.addnode for new host ganeti2046.codfw.wmnet to cluster codfw and group A 2449030 wikitext text/x-wiki hsd7z5cuckdcs5nf4ijq3ag7e07o0k5 2449031 2449030 2026-08-18T12:22:01Z Stashbot 7414 dbrant@deploy1003: helmfile [staging] START helmfile.d/services/wikifeeds: apply 2449031 wikitext text/x-wiki 5jmtd2elu39cso4d33tkndyzgtded61 2449032 2449031 2026-08-18T12:22:25Z Stashbot 7414 dbrant@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifeeds: apply 2449032 wikitext text/x-wiki ptwloednt8a8gzimk2xvn6ktunqpb1q 2449033 2449032 2026-08-18T12:22:48Z Stashbot 7414 dbrant@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifeeds: apply 2449033 wikitext text/x-wiki qq83yv9wr5wutow27sfa65th7yqd3et 2449035 2449033 2026-08-18T12:23:15Z Stashbot 7414 dbrant@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifeeds: apply 2449035 wikitext text/x-wiki e7or3o3simnnk0hfunxw2kheuekfbki 2449036 2449035 2026-08-18T12:23:26Z Stashbot 7414 dbrant@deploy1003: helmfile [codfw] START helmfile.d/services/wikifeeds: apply 2449036 wikitext text/x-wiki 1g9ndmcrmd16uve1lql9iaf1w3nl296 2449037 2449036 2026-08-18T12:23:53Z Stashbot 7414 dbrant@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply 2449037 wikitext text/x-wiki oc5h453zaz7vj32q9237j4xsqj0ibnj 2449038 2449037 2026-08-18T12:26:11Z Stashbot 7414 moritzm: readded ganeti2046 to the codfw cluster following firmware update and reimage T434681 2449038 wikitext text/x-wiki k7w8g1qcutw53pwgomjrnzmv7pltryw 2449039 2449038 2026-08-18T12:27:33Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.addnode (exit_code=0) for new host ganeti2046.codfw.wmnet to cluster codfw and group A 2449039 wikitext text/x-wiki tg0tkaxf27zdyzlvnqf0n0ulpengz65 2449040 2449039 2026-08-18T12:27:49Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1248-1261].eqiad.wmnet 2449040 wikitext text/x-wiki hekj1aliwt7qq6o2s92bp6xrnkgedz2 2449041 2449040 2026-08-18T12:28:47Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: helmfile [staging] START helmfile.d/services/wikidata-query-gui: apply 2449041 wikitext text/x-wiki fw1evl9174eclmamniqgwcbmrjmjmpc 2449042 2449041 2026-08-18T12:28:51Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2026.codfw.wmnet 2449042 wikitext text/x-wiki eeuk7j6xns9muwg0ho4d9q33ngdww2o 2449043 2449042 2026-08-18T12:29:25Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: helmfile [staging] DONE helmfile.d/services/wikidata-query-gui: apply 2449043 wikitext text/x-wiki tihjq1k5kmd0dlp4zv6ya3of0cv0cjk 2449044 2449043 2026-08-18T12:34:07Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: helmfile [codfw] START helmfile.d/services/wikidata-query-gui: apply 2449044 wikitext text/x-wiki n2envoiie9a59buhr9ncun0ej7rhcur 2449045 2449044 2026-08-18T12:34:29Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikidata-query-gui: apply 2449045 wikitext text/x-wiki piq85ft1t0bx35jqa9qp6ar2in4xwjr 2449046 2449045 2026-08-18T12:34:33Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: helmfile [eqiad] START helmfile.d/services/wikidata-query-gui: apply 2449046 wikitext text/x-wiki j8hry6hqhpnu8agyrbqvfn49u2xkx98 2449047 2449046 2026-08-18T12:34:52Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikidata-query-gui: apply 2449047 wikitext text/x-wiki 8qg0labeuwz5u208r0s42089ams1igb 2449048 2449047 2026-08-18T12:35:14Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2026.codfw.wmnet 2449048 wikitext text/x-wiki 52icfp3jfjhwmkca10w0jee0r1534lp 2449049 2449048 2026-08-18T12:36:10Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1248-1261].eqiad.wmnet 2449049 wikitext text/x-wiki pxla0bi5hqykoadeozcee5wouhqyt27 2449050 2449049 2026-08-18T12:36:17Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1248-1261].eqiad.wmnet 2449050 wikitext text/x-wiki 1zsex5jg0m6n413uq44q0lrwj1tyovy 2449051 2449050 2026-08-18T12:36:44Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1273,1275-1287].eqiad.wmnet 2449051 wikitext text/x-wiki 7bdyim2y050r2wvos0yjvz103d478hz 2449052 2449051 2026-08-18T12:37:28Z Stashbot 7414 dhinus: depool again clouddb10[13,14,16,18,20] that were repooled by the cookbook sre.mysql.multiinstance_reboot 2449052 wikitext text/x-wiki hitg7fq5w89qtuljjg7fbga7l501bxc 2449053 2449052 2026-08-18T12:37:35Z Stashbot 7414 fnegri@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1013.eqiad.wmnet 2449053 wikitext text/x-wiki 2ewgy3kojdm4utx736r7yztuxruhl60 2449054 2449053 2026-08-18T12:37:53Z Stashbot 7414 fnegri@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1014.eqiad.wmnet 2449054 wikitext text/x-wiki ek3o0qdqzwdb0zqzy0uopqr79dwgzfa 2449055 2449054 2026-08-18T12:38:11Z Stashbot 7414 fnegri@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1016.eqiad.wmnet 2449055 wikitext text/x-wiki a4f3fq4yyj671idroypruumegiluqgv 2449056 2449055 2026-08-18T12:38:17Z Stashbot 7414 fnegri@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1018.eqiad.wmnet 2449056 wikitext text/x-wiki ccugvbeed61212j6xtlbn8m5meqiadk 2449057 2449056 2026-08-18T12:38:23Z Stashbot 7414 fnegri@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1020.eqiad.wmnet 2449057 wikitext text/x-wiki la39nuldevhyh9k60m6u47ouk1d9jl5 2449058 2449057 2026-08-18T12:39:20Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host apt2002.wikimedia.org 2449058 wikitext text/x-wiki lr21ddpqxzprd074zi3qrgo9ihe4bl3 2449059 2449058 2026-08-18T12:39:46Z Stashbot 7414 fnegri@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1017.eqiad.wmnet 2449059 wikitext text/x-wiki 6efbpwxwiobgq0yo0ti58a43r12byeb 2449060 2449059 2026-08-18T12:40:22Z Stashbot 7414 fnegri@cumin1003: conftool action : set/pooled=yes; selector: name=clouddb1032 2449060 wikitext text/x-wiki e3nj6fv30gapgjcn0uby41mxvbq5623 2449061 2449060 2026-08-18T12:40:47Z Stashbot 7414 fnegri@cumin1003: conftool action : set/pooled=yes; selector: name=clouddb1032.eqiad.wmnet 2449061 wikitext text/x-wiki 81vsklsymekybawkjcepy4mkq6vowps 2449062 2449061 2026-08-18T12:40:55Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db2902.codfw.wmnet 2449062 wikitext text/x-wiki t6gx07fy57jfq5yum7nfbyf7r36e5mq 2449063 2449062 2026-08-18T12:40:58Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2449063 wikitext text/x-wiki 8g73w43irhl3554d7g4kkhpuic7b63u 2449065 2449063 2026-08-18T12:41:24Z Stashbot 7414 fnegri@cumin1003: conftool action : set/weight=100; selector: name=clouddb1032.eqiad.wmnet 2449065 wikitext text/x-wiki sz50383kvqw6jwwanzkfhh228lkq3ns 2449066 2449065 2026-08-18T12:41:47Z Stashbot 7414 dhinus: also depooled clouddb1017 (forgot it in the previous list) 2449066 wikitext text/x-wiki 6ql75z4z4vtaa3k3vie2ib4a4c27y7r 2449067 2449066 2026-08-18T12:42:11Z Stashbot 7414 dhinus: repooled clouddb1032 that was currently {"weight": 0, "pooled": "inactive"} for both s4 and s6 2449067 wikitext text/x-wiki pizdfbfw49a6j38hn5sazu1uc8cspn0 2449068 2449067 2026-08-18T12:44:39Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1273,1275-1287].eqiad.wmnet 2449068 wikitext text/x-wiki pwuvcmornh11wf3e3ca25k6op8qy8gp 2449069 2449068 2026-08-18T12:45:10Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host apt2002.wikimedia.org 2449069 wikitext text/x-wiki mwjugak5tg9tv7u07ukgoya7wfhsxnp 2449070 2449069 2026-08-18T12:46:53Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host apt1002.wikimedia.org 2449070 wikitext text/x-wiki nqnobtfkbbfiv5thhijoq6q4xf1izqc 2449071 2449070 2026-08-18T12:51:28Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db2902.codfw.wmnet - fceratto@cumin1003" 2449071 wikitext text/x-wiki q29l4kr6vwqt9v56fqqeohzshm38w9t 2449072 2449071 2026-08-18T12:51:32Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db2902.codfw.wmnet - fceratto@cumin1003" 2449072 wikitext text/x-wiki n0rud4p9iwqqiwmr02xc2tbszkooa8p 2449073 2449072 2026-08-18T12:51:35Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2449073 wikitext text/x-wiki b2svhzytd8l8egpxbokdpakjdoa4tcm 2449074 2449073 2026-08-18T12:51:38Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.wipe-cache db2902.codfw.wmnet on all recursors 2449074 wikitext text/x-wiki n9rydajmwuooafengsyyjl4d30f2fnp 2449075 2449074 2026-08-18T12:51:40Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) db2902.codfw.wmnet on all recursors 2449075 wikitext text/x-wiki roiodw0jayit9vqga3w24c2nw74kkn4 2449076 2449075 2026-08-18T12:52:46Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host apt1002.wikimedia.org 2449076 wikitext text/x-wiki cr7x1xhl9x23we0o5y0e6ipfuaq3ltq 2449077 2449076 2026-08-18T12:53:15Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1273,1275-1287].eqiad.wmnet 2449077 wikitext text/x-wiki eblzacfz8mjeyvnnuyjyv8oc3v85gsc 2449078 2449077 2026-08-18T12:53:24Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1273,1275-1287].eqiad.wmnet 2449078 wikitext text/x-wiki 0dmjkcrdoo4pwomh5nzbxehl6ndwat7 2449079 2449078 2026-08-18T12:53:51Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1288-1289,1291-1301].eqiad.wmnet 2449079 wikitext text/x-wiki rljn6su3zicnbv3n020w6qcfvuymgfx 2449080 2449079 2026-08-18T13:02:31Z Stashbot 7414 jgiannelos@deploy1003: Started scap sync-world: Backport for [[gerrit:1326232|prv: Enable parsoid rendering for 5 wikis (T435115)]] 2449080 wikitext text/x-wiki m9xay1k6iado2oxp63t7sk27s7teq7u 2449081 2449080 2026-08-18T13:02:35Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1288-1289,1291-1301].eqiad.wmnet 2449081 wikitext text/x-wiki mwazipfjyqwql80n41120zmbmce93zk 2449083 2449081 2026-08-18T13:04:48Z Stashbot 7414 jgiannelos@deploy1003: jgiannelos: Backport for [[gerrit:1326232|prv: Enable parsoid rendering for 5 wikis (T435115)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449083 wikitext text/x-wiki pyzfv2jpjzyr09p2bbl68lxzkzdqf1a 2449084 2449083 2026-08-18T13:05:35Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host rpki2003.codfw.wmnet 2449084 wikitext text/x-wiki b4eofkt0qe4qgipuv6h71slfjkw2vby 2449085 2449084 2026-08-18T13:06:30Z Stashbot 7414 jgiannelos@deploy1003: jgiannelos: Continuing with deployment 2449085 wikitext text/x-wiki 6lzb0mrhff4s1a4h16cyh5dffloxzk6 2449086 2449085 2026-08-18T13:09:21Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host rpki2003.codfw.wmnet 2449086 wikitext text/x-wiki o4f58js8ky6t3mxcsdajgdwe637l7tb 2449087 2449086 2026-08-18T13:10:16Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich-next: apply 2449087 wikitext text/x-wiki rf854rvb4digz1h1ka4ts3moaf2ezmw 2449088 2449087 2026-08-18T13:10:25Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1288-1289,1291-1301].eqiad.wmnet 2449088 wikitext text/x-wiki ec2tdd42zzgdv32er0g646h0squsjul 2449089 2449088 2026-08-18T13:10:29Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-feature-counts-change-enrich-next: apply 2449089 wikitext text/x-wiki hhd96zga67j9skbmn4b9woz2lezt1xn 2449090 2449089 2026-08-18T13:10:33Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1288-1289,1291-1301].eqiad.wmnet 2449090 wikitext text/x-wiki a99bnvqxu761my9zqxm0gkjljmkh01i 2449091 2449090 2026-08-18T13:10:48Z Stashbot 7414 jgiannelos@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326232|prv: Enable parsoid rendering for 5 wikis (T435115)]] (duration: 08m 17s) 2449091 wikitext text/x-wiki k7ft81dpa9br4lmieychbsj8zxcvy9u 2449092 2449091 2026-08-18T13:11:03Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1302-1314].eqiad.wmnet 2449092 wikitext text/x-wiki tolk14byhpfzvam1kfgbe5ya3t3alw6 2449095 2449092 2026-08-18T13:13:21Z Stashbot 7414 moritzm: installing util-linux security updates 2449095 wikitext text/x-wiki gqib3w5gayw1jhppvws6tc1go6z8dmu 2449096 2449095 2026-08-18T13:14:11Z Stashbot 7414 stran@deploy1003: Started scap sync-world: Backport for [[gerrit:1326775|SI: Instrument case update on first edit (T435048)]] 2449096 wikitext text/x-wiki f61t1281jplfs5o0xdxwl3a16qgmb40 2449098 2449096 2026-08-18T13:16:13Z Stashbot 7414 stran@deploy1003: stran: Backport for [[gerrit:1326775|SI: Instrument case update on first edit (T435048)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449098 wikitext text/x-wiki o8qn2uvm0m7t9tgk7d8m2tggq4varjw 2449099 2449098 2026-08-18T13:17:09Z Stashbot 7414 stran@deploy1003: stran: Continuing with deployment 2449099 wikitext text/x-wiki 92gi38mrs7mxkjxb5pf1bka751lij8d 2449100 2449099 2026-08-18T13:18:32Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1302-1314].eqiad.wmnet 2449100 wikitext text/x-wiki m8qooygikrzuizaxxy4c1svkv2uoaoq 2449101 2449100 2026-08-18T13:21:24Z Stashbot 7414 stran@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326775|SI: Instrument case update on first edit (T435048)]] (duration: 07m 12s) 2449101 wikitext text/x-wiki b7520jpgclb4t6pa16stmvlef343xyj 2449102 2449101 2026-08-18T13:27:00Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1302-1314].eqiad.wmnet 2449102 wikitext text/x-wiki 33shtwmev9hydwo47ma6reo7u6mqjs4 2449103 2449102 2026-08-18T13:27:08Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1302-1314].eqiad.wmnet 2449103 wikitext text/x-wiki tsd3s0k97tluzeple4g5o5vlkhoczfm 2449104 2449103 2026-08-18T13:27:35Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1315-1327].eqiad.wmnet 2449104 wikitext text/x-wiki q0apilb97pi36s204atv0cchbnb53i3 2449107 2449104 2026-08-18T13:34:49Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host rpki1001.eqiad.wmnet 2449107 wikitext text/x-wiki gzcqo9ki15erhjxhdfi0ujkzb8kxtkh 2449108 2449107 2026-08-18T13:35:11Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1315-1327].eqiad.wmnet 2449108 wikitext text/x-wiki r1fe0khp8slz8ock9m1bra1vbkor1wt 2449109 2449108 2026-08-18T13:40:15Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host rpki1001.eqiad.wmnet 2449109 wikitext text/x-wiki chkiq1tjt2s683nfx1vt2ekwnrxvrqj 2449111 2449109 2026-08-18T13:43:40Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1315-1327].eqiad.wmnet 2449111 wikitext text/x-wiki l0513b5jmnps3z9sa951hlx7k4ha0ev 2449112 2449111 2026-08-18T13:43:46Z Stashbot 7414 cdanis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-main: apply 2449112 wikitext text/x-wiki 3nsisx38n8s1i7nsu94gcosyndvltxg 2449113 2449112 2026-08-18T13:43:48Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1315-1327].eqiad.wmnet 2449113 wikitext text/x-wiki 03gl4jkoekz97d98pqefjrvgvu1m7nz 2449114 2449113 2026-08-18T13:43:51Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker-exp1001.eqiad.wmnet 2449114 wikitext text/x-wiki 19b6et7qvzfwl4px1qh6p7cc7rd6bkw 2449115 2449114 2026-08-18T13:44:18Z Stashbot 7414 cdanis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-main: apply 2449115 wikitext text/x-wiki jpqr1iziim9w15tb4e6accohdu8awoe 2449116 2449115 2026-08-18T13:44:23Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker-exp1001.eqiad.wmnet 2449116 wikitext text/x-wiki nc1o2vaaq9d35swr9k5ly71v575zbob 2449118 2449116 2026-08-18T13:47:10Z Stashbot 7414 stran@deploy1003: Started scap sync-world: Backport for [[gerrit:1326838|Enable Suggested Investigations on hewiki (T435146)]] 2449118 wikitext text/x-wiki 6tbq7mp973oxjrdu4xsmuqm4gr3zz1r 2449120 2449118 2026-08-18T13:47:53Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-analytics: apply 2449120 wikitext text/x-wiki jv7zm77yc0dxxqk0kagfd2w2e3yc5l4 2449121 2449120 2026-08-18T13:47:57Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host puppetboard2003.codfw.wmnet 2449121 wikitext text/x-wiki izhphpgg89rzr39jfnjd8hfkalsm44v 2449122 2449121 2026-08-18T13:48:04Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-analytics: apply 2449122 wikitext text/x-wiki m6c4b0ja61spsjn470q6lgzhjizlgsg 2449123 2449122 2026-08-18T13:48:39Z Stashbot 7414 phuedx@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-analytics: apply 2449123 wikitext text/x-wiki pdaey8xc6jd9dw5k55t280hyxoowm1x 2449124 2449123 2026-08-18T13:49:18Z Stashbot 7414 stran@deploy1003: stran: Backport for [[gerrit:1326838|Enable Suggested Investigations on hewiki (T435146)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449124 wikitext text/x-wiki 4hujlxu4iavwsowp06k0jfbaga8k3xn 2449125 2449124 2026-08-18T13:49:24Z Stashbot 7414 phuedx@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-analytics: apply 2449125 wikitext text/x-wiki iz8x1x7v1iqno21m2vrojc3tm0t6fm0 2449126 2449125 2026-08-18T13:50:12Z Stashbot 7414 phuedx@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-analytics: apply 2449126 wikitext text/x-wiki kk5a3hbm74ll4947hkrdrl2lsheet97 2449127 2449126 2026-08-18T13:50:32Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker-exp1001.eqiad.wmnet 2449127 wikitext text/x-wiki aytjflqhrr137mwj2ccda5dcx9wcuxt 2449128 2449127 2026-08-18T13:50:35Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker-exp1001.eqiad.wmnet 2449128 wikitext text/x-wiki tfa3egnl7kx4fk8434sh66peru3vdsw 2449129 2449128 2026-08-18T13:50:38Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on A:wikikube-worker-eqiad 2449129 wikitext text/x-wiki qmfl2w1b22222e6ficuaxwvqe9cvh9d 2449130 2449129 2026-08-18T13:50:54Z Stashbot 7414 phuedx@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics: apply 2449130 wikitext text/x-wiki d5m89uyz5t2z9oaxwq1imjb5mi6vhtu 2449131 2449130 2026-08-18T13:51:35Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host puppetboard2003.codfw.wmnet 2449131 wikitext text/x-wiki jut2s97bjiveeygwpuadv3zyaojji3f 2449132 2449131 2026-08-18T13:51:44Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2026.codfw.wmnet 2449132 wikitext text/x-wiki dib1avjiucsu6hfi8dx9jtviuifg62r 2449133 2449132 2026-08-18T13:51:50Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM db2902.codfw.wmnet - fceratto@cumin1003" 2449133 wikitext text/x-wiki lmiq3v0k08pksfaxk2ekjdl33tc3yy0 2449134 2449133 2026-08-18T13:51:54Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM db2902.codfw.wmnet - fceratto@cumin1003" 2449134 wikitext text/x-wiki h5dtowky3xghq374ylml6o0bhbk5wrd 2449135 2449134 2026-08-18T13:53:01Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2041.codfw.wmnet 2449135 wikitext text/x-wiki 7hlerj7125hvzeejm9mwqm9qx9vl3bt 2449136 2449135 2026-08-18T13:53:18Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host puppetboard1003.eqiad.wmnet 2449136 wikitext text/x-wiki amva8ymroslja6aqd0uv5kl15osquk8 2449137 2449136 2026-08-18T13:53:53Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-logging-external: apply 2449137 wikitext text/x-wiki rxbi0dv46her3qpdhesgvbiwnv5d7p8 2449138 2449137 2026-08-18T13:54:02Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-logging-external: apply 2449138 wikitext text/x-wiki a07jprnn88ewyyu8mx42cbh3246jx6g 2449139 2449138 2026-08-18T13:54:11Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-logging-external: apply 2449139 wikitext text/x-wiki pmbdwbncobp01dh8tlbhg7oa44z6pfq 2449140 2449139 2026-08-18T13:54:21Z Stashbot 7414 phuedx@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-logging-external: apply 2449140 wikitext text/x-wiki nzo6k2vmg1xscrsptx08l7f20z9jk4g 2449141 2449140 2026-08-18T13:54:40Z Stashbot 7414 stran@deploy1003: stran: Continuing with deployment 2449141 wikitext text/x-wiki 0znrgzsosfnxwmj5ie32n1qqfs0f3y5 2449142 2449141 2026-08-18T13:54:49Z Stashbot 7414 phuedx@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-logging-external: apply 2449142 wikitext text/x-wiki f76l5264lj7fieg8kc4du3egxboaago 2449143 2449142 2026-08-18T13:55:42Z Stashbot 7414 phuedx@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-logging-external: apply 2449143 wikitext text/x-wiki gnelxbr4rondimvhxt2l18wic8v13pg 2449144 2449143 2026-08-18T13:55:53Z Stashbot 7414 phuedx@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-logging-external: apply 2449144 wikitext text/x-wiki iivlc4jb8g9bikan0b5hv82jof3dy8e 2449145 2449144 2026-08-18T13:57:05Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-main: apply 2449145 wikitext text/x-wiki 6x1sgdx3h85ea01iex5p36p857wzzw1 2449146 2449145 2026-08-18T13:57:08Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host puppetboard1003.eqiad.wmnet 2449146 wikitext text/x-wiki g7ev0pc2s4ul907pjkyc5aqo0f8nd7m 2449147 2449146 2026-08-18T13:57:14Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-main: apply 2449147 wikitext text/x-wiki 431ic98fikq88mqm24q8i9k9qnuwonx 2449148 2449147 2026-08-18T13:57:50Z Stashbot 7414 phuedx@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-main: apply 2449148 wikitext text/x-wiki jwuvl6asjgq02cuucf9wrv99excdgr4 2449149 2449148 2026-08-18T13:58:36Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2041.codfw.wmnet 2449149 wikitext text/x-wiki j5b6lengvca3gbv5vktud7kc6rf08jf 2449150 2449149 2026-08-18T13:58:39Z Stashbot 7414 phuedx@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-main: apply 2449150 wikitext text/x-wiki k8fa8k0oe1haugdz4fz8vywabaqs125 2449151 2449150 2026-08-18T13:59:01Z Stashbot 7414 stran@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326838|Enable Suggested Investigations on hewiki (T435146)]] (duration: 11m 50s) 2449151 wikitext text/x-wiki 0ijqx9qwi9cwtn15htqylonj98d3cgh 2449152 2449151 2026-08-18T13:59:19Z Stashbot 7414 phuedx@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-main: apply 2449152 wikitext text/x-wiki h2t4wfuhq9zgph1o09lwg38lvkgrw60 2449153 2449152 2026-08-18T13:59:58Z Stashbot 7414 phuedx@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-main: apply 2449153 wikitext text/x-wiki iy8bfwist7x9ujhmj9u2n8ezcnwqx4q 2449154 2449153 2026-08-18T14:04:02Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2041.codfw.wmnet 2449154 wikitext text/x-wiki jufx70yl34ndebrp916ymc2vf0ittpl 2449155 2449154 2026-08-18T14:04:31Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2041.codfw.wmnet 2449155 wikitext text/x-wiki 1k32ofrbzsid9hqvr0qflebkb69jjd7 2449156 2449155 2026-08-18T14:04:56Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2042.codfw.wmnet 2449156 wikitext text/x-wiki 8gr2nv7wvxdum6ltwqn4utv7jq72am6 2449157 2449156 2026-08-18T14:16:43Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2042.codfw.wmnet 2449157 wikitext text/x-wiki doczo9kv99h9oi1l9yk407zgpi68dfq 2449158 2449157 2026-08-18T14:16:52Z Stashbot 7414 elukey: uploaded spicerack_13.2.0 to apt.wikimedia.org bookworm-wikimedia,trixie-wikimedia 2449158 wikitext text/x-wiki 0st4c3ybts2nus3o8bam0n8d8lszg3g 2449159 2449158 2026-08-18T14:21:27Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.reimage for host db2902.codfw.wmnet with OS trixie 2449159 wikitext text/x-wiki eanbe3wq11lhqmr5hbo64yoyk7b195n 2449160 2449159 2026-08-18T14:22:03Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2042.codfw.wmnet 2449160 wikitext text/x-wiki 5v3608bdxbaxepxfauv6lu8xlx2vf7h 2449161 2449160 2026-08-18T14:22:10Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2042.codfw.wmnet 2449161 wikitext text/x-wiki 88u4s147jz38c8pax1kfi6sjyz0ggzb 2449164 2449161 2026-08-18T14:33:49Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2043.codfw.wmnet 2449164 wikitext text/x-wiki m1yuo72999ymu8uq3ek3qwzimlkk1i1 2449169 2449164 2026-08-18T14:45:42Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2043.codfw.wmnet 2449169 wikitext text/x-wiki 4kotbkehn4abmu2ihpj5ixwofhtiqim 2449170 2449169 2026-08-18T14:51:02Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2043.codfw.wmnet 2449170 wikitext text/x-wiki 6x3eqj30yczoz8eo2brq725hoif8sgk 2449171 2449170 2026-08-18T14:51:10Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2043.codfw.wmnet 2449171 wikitext text/x-wiki sarlbi4c8z380wmjsyjrem4vmkb3zjn 2449172 2449171 2026-08-18T14:53:58Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1236.eqiad.wmnet with OS bookworm 2449172 wikitext text/x-wiki pk2jkuxztlooluho8uoiijy98gwz26u 2449173 2449172 2026-08-18T14:55:51Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/growthbook-next: apply 2449173 wikitext text/x-wiki 5xcsf8unnmcudy0c6vkdhciuilfe25f 2449174 2449173 2026-08-18T14:56:52Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2044.codfw.wmnet 2449174 wikitext text/x-wiki a9hto9xyeubtrghgxnhi5ikwugqprpu 2449175 2449174 2026-08-18T14:57:02Z Stashbot 7414 arnaudb@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on phab2003.codfw.wmnet,phab[1004-1006].eqiad.wmnet with reason: maintenance 2449175 wikitext text/x-wiki 0s9t956ui8higzghhap9lsrwnjd7y7e 2449176 2449175 2026-08-18T14:57:04Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/growthboo-next: apply 2449176 wikitext text/x-wiki s61u1g3s7foy6oh65bo2ei9aprqj6cu 2449177 2449176 2026-08-18T15:02:11Z Stashbot 7414 brennen@deploy1003: Started deploy [phabricator/deployment@6b9b6ff]: deploy phab2003 for T435213 2449177 wikitext text/x-wiki 5d4zraom3kfqhfcb2p9tejy2s1kipuv 2449178 2449177 2026-08-18T15:03:07Z Stashbot 7414 brennen@deploy1003: Finished deploy [phabricator/deployment@6b9b6ff]: deploy phab2003 for T435213 (duration: 00m 57s) 2449178 wikitext text/x-wiki mtz69tsx8dk1fd56nt64y81bncjpttq 2449179 2449178 2026-08-18T15:03:23Z Stashbot 7414 brennen@deploy1003: Started deploy [phabricator/deployment@6b9b6ff]: deploy phab1004 for T435213 2449179 wikitext text/x-wiki im050x9il0rxoy5cemfgm74nas7pocl 2449180 2449179 2026-08-18T15:03:29Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2044.codfw.wmnet 2449180 wikitext text/x-wiki ffgmf990zz576uwp7at43x3jvcghw8p 2449181 2449180 2026-08-18T15:04:26Z Stashbot 7414 brennen@deploy1003: Finished deploy [phabricator/deployment@6b9b6ff]: deploy phab1004 for T435213 (duration: 01m 01s) 2449181 wikitext text/x-wiki qy7ykq4r1zr2iejhzntrwi8ipvn3hdp 2449182 2449181 2026-08-18T15:08:50Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2044.codfw.wmnet 2449182 wikitext text/x-wiki g9x9hjs9g4jt4570k13cc5lgryl6psh 2449183 2449182 2026-08-18T15:09:08Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host build2002.codfw.wmnet 2449183 wikitext text/x-wiki 6wrzbbs8rwnbqw971yki3rgrfvzo87q 2449184 2449183 2026-08-18T15:09:15Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2044.codfw.wmnet 2449184 wikitext text/x-wiki 2ovop47nzftn2366e5fca2n9bzjrakm 2449185 2449184 2026-08-18T15:09:51Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1236.eqiad.wmnet with reason: host reimage 2449185 wikitext text/x-wiki 4uee0vemyb92j1mgbd7xmxtv44q7gbf 2449186 2449185 2026-08-18T15:12:41Z Stashbot 7414 moritzm: failover ganeti master in codfw to ganeti2047 2449186 wikitext text/x-wiki 8vb984za1aq6n8gbwsbbdszo6zvnunk 2449187 2449186 2026-08-18T15:13:24Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1236.eqiad.wmnet with reason: host reimage 2449187 wikitext text/x-wiki nrnc4f1vomoyipwmfx6ohlfvyxafjbl 2449188 2449187 2026-08-18T15:14:59Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host build2002.codfw.wmnet 2449188 wikitext text/x-wiki 9vkexv31sywnoj28l2y6f7dkwqpiud7 2449189 2449188 2026-08-18T15:20:45Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-staging-codfw] 'sync' command on namespace 'ores-legacy' for release 'main' . 2449189 wikitext text/x-wiki olck15lzizl2lihiwny7atlmefr4c7p 2449192 2449189 2026-08-18T15:26:40Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-codfw] 'sync' command on namespace 'ores-legacy' for release 'main' . 2449192 wikitext text/x-wiki l4ov2vtublb8v84ri5r7dc49m9ftmer 2449193 2449192 2026-08-18T15:29:27Z Stashbot 7414 aikochou@deploy1003: helmfile [ml-serve-eqiad] 'sync' command on namespace 'ores-legacy' for release 'main' . 2449193 wikitext text/x-wiki 6dhb9hwtes6o25q7jxfzqx9f4dxew7j 2449194 2449193 2026-08-18T15:37:15Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1236.eqiad.wmnet with OS bookworm 2449194 wikitext text/x-wiki c9frbt5nbje4358y08dya3mioj0ijzf 2449195 2449194 2026-08-18T15:41:06Z Stashbot 7414 topranks: bounce PIC 0/0 on cr1-magru to set port to 40G 2449195 wikitext text/x-wiki pwdea0rr2lnqxut1vemm6tcmgcbkowv 2449196 2449195 2026-08-18T15:47:56Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1145.eqiad.wmnet with OS bookworm 2449196 wikitext text/x-wiki h1yvi0o0wjudcyx0lxgdwops40x2pgn 2449197 2449196 2026-08-18T15:47:59Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1199.eqiad.wmnet with OS bookworm 2449197 wikitext text/x-wiki 99847cgkqnr4qin92bal25292f9k1o6 2449198 2449197 2026-08-18T15:48:04Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1200.eqiad.wmnet with OS bookworm 2449198 wikitext text/x-wiki on0unbfou4zvglbo9z7r638dihhlnof 2449200 2449198 2026-08-18T15:50:26Z Stashbot 7414 moritzm: installing zip security updates 2449200 wikitext text/x-wiki 5fw3wfm0ewp7h27u0drwa9l3e9xc788 2449206 2449200 2026-08-18T16:02:27Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1199.eqiad.wmnet with reason: host reimage 2449206 wikitext text/x-wiki p86v95qyl1bmesqmx8sr675dmx2slfh 2449207 2449206 2026-08-18T16:02:57Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1145.eqiad.wmnet with reason: host reimage 2449207 wikitext text/x-wiki 6rjwwgb2oqnpkhctuyr9yjl639d5eaq 2449208 2449207 2026-08-18T16:03:45Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1200.eqiad.wmnet with reason: host reimage 2449208 wikitext text/x-wiki 0jdc56jekgjb78p39859ditt1e3u71y 2449209 2449208 2026-08-18T16:05:29Z Stashbot 7414 cjd91: sudo -i cookbook sre.cdn.roll-upgrade-ats --query 'A:cp-esams' --task-id T434478 --reason '9.2.15 upgrade' 2449209 wikitext text/x-wiki ofzqsyvlqal2l5uq02l6bf3r0f93y3q 2449210 2449209 2026-08-18T16:05:38Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-upgrade-ats Rolling upgrade of ATS on A:cp-esams and A:cp - 9.2.15 upgrade (T434478) 2449210 wikitext text/x-wiki kmfv4d0o9ytzch82ljx6alfe95aa6bi 2449214 2449210 2026-08-18T16:09:46Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1199.eqiad.wmnet with reason: host reimage 2449214 wikitext text/x-wiki b3ji39t0idm58mtrjnvxm7mq2sz93fz 2449216 2449214 2026-08-18T16:13:45Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1200.eqiad.wmnet with reason: host reimage 2449216 wikitext text/x-wiki 5i8bvuvfplod6efwrcw20vpr0t62xj7 2449217 2449216 2026-08-18T16:18:21Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1145.eqiad.wmnet with reason: host reimage 2449217 wikitext text/x-wiki 28ospeku5l1tpreo7i95e4e63nja3a0 2449220 2449217 2026-08-18T16:32:51Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1199.eqiad.wmnet with OS bookworm 2449220 wikitext text/x-wiki i9d5hst0cy58iqryngl0u2g6ecbe5lb 2449223 2449220 2026-08-18T16:34:14Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1200.eqiad.wmnet with OS bookworm 2449223 wikitext text/x-wiki dz4p2gze3qpmvalwjx9dzt4oxv4oxf8 2449226 2449223 2026-08-18T16:41:21Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1145.eqiad.wmnet with OS bookworm 2449226 wikitext text/x-wiki 6m245047h7s1gezrwnm3scdmr3dj38v 2449227 2449226 2026-08-18T16:41:38Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.282.0" for 3 host(s) 2449227 wikitext text/x-wiki nimqp3h7vgeeiln74hqdexg429syeq7 2449229 2449227 2026-08-18T16:43:33Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.282.0" completed for 3 hosts 2449229 wikitext text/x-wiki kt3u0kqadzedctar85wa0jpyam68n67 2449231 2449229 2026-08-18T16:43:54Z Stashbot 7414 dancy@deploy1003: Started scap sync-world: Testing T434726 2449231 wikitext text/x-wiki 0txr035n42rzsh4x2h5t47f34981wni 2449234 2449231 2026-08-18T16:50:34Z Stashbot 7414 dancy@deploy1003: Finished scap sync-world: Testing T434726 (duration: 06m 40s) 2449234 wikitext text/x-wiki 2hht7yo8gk5sffhbhs3j8y88zijyuwj 2449237 2449234 2026-08-18T17:02:22Z Stashbot 7414 dancy@deploy1003: Started scap sync-world: Backport for [[gerrit:1326885|w/deployment-info.php: Handle new file format (T434726)]] 2449237 wikitext text/x-wiki 9py037jkeuakag6k3wheriieu4zhpg8 2449238 2449237 2026-08-18T17:04:38Z Stashbot 7414 dancy@deploy1003: dancy: Backport for [[gerrit:1326885|w/deployment-info.php: Handle new file format (T434726)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449238 wikitext text/x-wiki 92g1rwjaouvyswwzlmdzjt28t8uwi5n 2449239 2449238 2026-08-18T17:05:23Z Stashbot 7414 dancy@deploy1003: dancy: Continuing with deployment 2449239 wikitext text/x-wiki 8uq57t7qy3j6rs3dfxs0if35o9izwem 2449240 2449239 2026-08-18T17:08:38Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2449240 wikitext text/x-wiki have5igcktaojcb3il7m8hojb9os7s9 2449241 2449240 2026-08-18T17:09:37Z Stashbot 7414 dancy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326885|w/deployment-info.php: Handle new file format (T434726)]] (duration: 07m 15s) 2449241 wikitext text/x-wiki 4nnnqmaixo8ar51jrbh5lm3y70i9rnx 2449244 2449241 2026-08-18T17:14:05Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1146.eqiad.wmnet with OS bookworm 2449244 wikitext text/x-wiki 2f0xl5b1qcxifhq0kc5i8n6jtgwer1r 2449245 2449244 2026-08-18T17:14:07Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1156.eqiad.wmnet with OS bookworm 2449245 wikitext text/x-wiki osry3vyhtxumhk35vncmom2l03r3z3q 2449246 2449245 2026-08-18T17:14:10Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1177.eqiad.wmnet with OS bookworm 2449246 wikitext text/x-wiki nt6339ttsz5b8032ulq83r5511tpzsa 2449247 2449246 2026-08-18T17:14:13Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1201.eqiad.wmnet with OS bookworm 2449247 wikitext text/x-wiki 08n4fqvxsw6t21ziac1z9bmzr851ymx 2449248 2449247 2026-08-18T17:14:16Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1202.eqiad.wmnet with OS bookworm 2449248 wikitext text/x-wiki 4yyytdql1p2elovtct4fg3usejvmyxq 2449249 2449248 2026-08-18T17:21:59Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage 2449249 wikitext text/x-wiki 25j1w4kvf92saxcc8ybklnmpoz92boq 2449251 2449249 2026-08-18T17:26:00Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage 2449251 wikitext text/x-wiki 0y1mcyg3infksh6h96lg94ycmoa931t 2449252 2449251 2026-08-18T17:27:56Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1201.eqiad.wmnet with reason: host reimage 2449252 wikitext text/x-wiki 6z9cyimf1d4ifak9zfcxcm6lk7xbbth 2449253 2449252 2026-08-18T17:28:02Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1202.eqiad.wmnet with reason: host reimage 2449253 wikitext text/x-wiki dlyvoi14rzx92n54zc0vgzp394vd8n5 2449254 2449253 2026-08-18T17:28:22Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1146.eqiad.wmnet with reason: host reimage 2449254 wikitext text/x-wiki k1hyv9lzambmgudg5jcz92mtozeorod 2449255 2449254 2026-08-18T17:29:33Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1177.eqiad.wmnet with reason: host reimage 2449255 wikitext text/x-wiki qiga7etnjymelcz89zl5egvn4iq8qi8 2449256 2449255 2026-08-18T17:30:03Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1156.eqiad.wmnet with reason: host reimage 2449256 wikitext text/x-wiki 6h9aa4ybfkucxnb7ye486xbcnjq37ha 2449259 2449256 2026-08-18T17:34:49Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1201.eqiad.wmnet with reason: host reimage 2449259 wikitext text/x-wiki cz2rt8qpuc194y8dn0fy5ss5jfvsfj7 2449261 2449259 2026-08-18T17:38:04Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1202.eqiad.wmnet with reason: host reimage 2449261 wikitext text/x-wiki pab7kyrhpffjmhlqye4zz8ls0xaclh3 2449263 2449261 2026-08-18T17:41:56Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1177.eqiad.wmnet with reason: host reimage 2449263 wikitext text/x-wiki f0l9we9pju9tj7yv1u48195uabh80g6 2449264 2449263 2026-08-18T17:42:34Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2449264 wikitext text/x-wiki 14c1upr3fidlrobowt9nbafx0b9dk8q 2449265 2449264 2026-08-18T17:45:33Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on A:cp-esams and A:cp - 9.2.15 upgrade (T434478) 2449265 wikitext text/x-wiki 97vav1czkg7cw3fv0jaguh2i9q2ahea 2449266 2449265 2026-08-18T17:46:06Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1146.eqiad.wmnet with reason: host reimage 2449266 wikitext text/x-wiki tj8v1yj3ngkf4lmjzfodc7boj3oc13j 2449267 2449266 2026-08-18T17:48:50Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1156.eqiad.wmnet with reason: host reimage 2449267 wikitext text/x-wiki joingqzw8fkl0cz5atxjqnqn5i820vy 2449268 2449267 2026-08-18T17:48:59Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1326380|Introduce main lock manager service (T366938 T427999)]] 2449268 wikitext text/x-wiki kzd0wq3kl5xzgxpknmd8oaupuykgy2w 2449269 2449268 2026-08-18T17:51:00Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1326380|Introduce main lock manager service (T366938 T427999)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449269 wikitext text/x-wiki 9y6fp9ue68gb8mxpr41zb51nfie8avl 2449271 2449269 2026-08-18T17:55:28Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1201.eqiad.wmnet with OS bookworm 2449271 wikitext text/x-wiki 6hjv4ggni4na52id6s5f2x1h9fhja0s 2449272 2449271 2026-08-18T17:58:38Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1202.eqiad.wmnet with OS bookworm 2449272 wikitext text/x-wiki mjnk6hd64ny0eyxd7efaduenw6s9pq8 2449273 2449272 2026-08-18T17:58:45Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Rolling back deployment 2449273 wikitext text/x-wiki 0wtl0va8jx9eri69agttrq1s5c2d1ze 2449276 2449273 2026-08-18T18:00:23Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326380|Introduce main lock manager service (T366938 T427999)]] (duration: 11m 25s) 2449276 wikitext text/x-wiki 3cpj6whoqx8bhz7yixgigphscrs3lvf 2449278 2449276 2026-08-18T18:07:56Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1177.eqiad.wmnet with OS bookworm 2449278 wikitext text/x-wiki thde5yp2hfx8oh86s6n0n9c93uvlsa4 2449279 2449278 2026-08-18T18:08:43Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1146.eqiad.wmnet with OS bookworm 2449279 wikitext text/x-wiki ostyuql5lo6alru4azo0gsws11r5w5f 2449282 2449279 2026-08-18T18:11:58Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply 2449282 wikitext text/x-wiki jj90iqlxfi21bri42yzg66azymtjpw6 2449283 2449282 2026-08-18T18:12:32Z Stashbot 7414 aklapper@deploy1003: Started scap sync-world: Backport for [[gerrit:1326852|CategoryTree: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]], [[gerrit:1326853|CategoryViewer: Allow null $html in the CategoryViewerGenerateLink hook (T435161)]], [[gerrit:1326865|Flow: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]] 2449283 wikitext text/x-wiki snpu47y8nxsbnml3xmyhpvn37l74h8i 2449284 2449283 2026-08-18T18:12:40Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply 2449284 wikitext text/x-wiki 93xzeb8s9rvlbh9w2ifyjr7epftp85g 2449285 2449284 2026-08-18T18:13:27Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1156.eqiad.wmnet with OS bookworm 2449285 wikitext text/x-wiki r0b6bofctow1zzx03imfho98bp34p27 2449286 2449285 2026-08-18T18:14:38Z Stashbot 7414 aklapper@deploy1003: jforrester, aklapper: Backport for [[gerrit:1326852|CategoryTree: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]], [[gerrit:1326853|CategoryViewer: Allow null $html in the CategoryViewerGenerateLink hook (T435161)]], [[gerrit:1326865|Flow: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki 2449286 wikitext text/x-wiki 02y8fanwuql58wlyn6nor05mvdpkkez 2449287 2449286 2026-08-18T18:17:50Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.reimage for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2449287 wikitext text/x-wiki dlp3i66ga1p3se7ybg2gyrjv1u4q6mz 2449288 2449287 2026-08-18T18:18:12Z Stashbot 7414 aklapper@deploy1003: jforrester, aklapper: Continuing with deployment 2449288 wikitext text/x-wiki 132du01e9t8c267139d4w0tdtbj11um 2449289 2449288 2026-08-18T18:22:29Z Stashbot 7414 aklapper@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326852|CategoryTree: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]], [[gerrit:1326853|CategoryViewer: Allow null $html in the CategoryViewerGenerateLink hook (T435161)]], [[gerrit:1326865|Flow: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]] (duration: 09m 57s) 2449289 wikitext text/x-wiki qtjgu3g8xq9xxdds779bl3kyriao7gn 2449291 2449289 2026-08-18T18:31:04Z Stashbot 7414 jasmine@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage 2449291 wikitext text/x-wiki 5w7ipqnbec2ang6bzu6m4wqssx3s00y 2449292 2449291 2026-08-18T18:34:39Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage 2449292 wikitext text/x-wiki lt24p96srnmlpw2jg8hw93fddngqk3f 2449293 2449292 2026-08-18T18:44:26Z Stashbot 7414 aklapper@deploy1003: rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.16 refs T430835 2449293 wikitext text/x-wiki cz4f7eke873s35ygma9o73zvv7mzst8 2449296 2449293 2026-08-18T18:50:17Z Stashbot 7414 jasmine@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-ctrl2006.codfw.wmnet with OS trixie 2449296 wikitext text/x-wiki eger9kobiws1ims6gryf8q465cy903u 2449298 2449296 2026-08-18T18:54:44Z Stashbot 7414 reedy@deploy1003: Started scap sync-world: Backport for [[gerrit:1324751|InitialiseSettings: Enable 2FA enforcement on more private wikis (T428103)]], [[gerrit:1326875|Add banner notifying of upcoming 2FA enforcement (T420792)]] 2449298 wikitext text/x-wiki l48xijf6oiqlpw6jucimc29cw1bgu3j 2449301 2449298 2026-08-18T19:12:57Z Stashbot 7414 reedy@deploy1003: reedy: Backport for [[gerrit:1324751|InitialiseSettings: Enable 2FA enforcement on more private wikis (T428103)]], [[gerrit:1326875|Add banner notifying of upcoming 2FA enforcement (T420792)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449301 wikitext text/x-wiki b502gvjii8xx03pg4qdc33mnn4slbyg 2449302 2449301 2026-08-18T19:13:17Z Stashbot 7414 reedy@deploy1003: reedy: Continuing with deployment 2449302 wikitext text/x-wiki hjeiha86k45n675uve5k3p5fpxbspsw 2449303 2449302 2026-08-18T19:14:29Z Stashbot 7414 denisse: rebooting kafkamon1003.eqiad.wmnet T435162 2449303 wikitext text/x-wiki dqokrai237qjgb591r5zkkpptezwlzy 2449304 2449303 2026-08-18T19:15:42Z Stashbot 7414 denisse: rebooting kafkamon2003.codfw.wmnet - T435162 2449304 wikitext text/x-wiki d7n4fra93exup9dfz66fhi8xq60flzi 2449305 2449304 2026-08-18T19:16:44Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1166.eqiad.wmnet with OS bookworm 2449305 wikitext text/x-wiki c0ooy4qj89o9n4delz6hew1v2zmckya 2449306 2449305 2026-08-18T19:16:48Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1167.eqiad.wmnet with OS bookworm 2449306 wikitext text/x-wiki 4nw76lypdj2p73ibwuao6ts40zx7ylu 2449307 2449306 2026-08-18T19:16:53Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1168.eqiad.wmnet with OS bookworm 2449307 wikitext text/x-wiki kygdcswcg2a1yo7jebyywz4ww1j3jzp 2449308 2449307 2026-08-18T19:16:56Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1203.eqiad.wmnet with OS bookworm 2449308 wikitext text/x-wiki kh9dd9u7cg0jc78v5c6xlbqexb074gp 2449309 2449308 2026-08-18T19:17:00Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1204.eqiad.wmnet with OS bookworm 2449309 wikitext text/x-wiki mmxbiqv9pd5uydytsfou8zi3upwpqcg 2449310 2449309 2026-08-18T19:26:32Z Stashbot 7414 reedy@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324751|InitialiseSettings: Enable 2FA enforcement on more private wikis (T428103)]], [[gerrit:1326875|Add banner notifying of upcoming 2FA enforcement (T420792)]] (duration: 31m 46s) 2449310 wikitext text/x-wiki tdf71qlvdv3p49wgmc55q5cdtnwpsmj 2449311 2449310 2026-08-18T19:31:01Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1203.eqiad.wmnet with reason: host reimage 2449311 wikitext text/x-wiki 8ch6alh2mlxr9q2w0ko9chz7j3b7jze 2449312 2449311 2026-08-18T19:31:16Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1204.eqiad.wmnet with reason: host reimage 2449312 wikitext text/x-wiki ejuqmglpyt464fmlma1w0j9u95lyubu 2449313 2449312 2026-08-18T19:32:13Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1166.eqiad.wmnet with reason: host reimage 2449313 wikitext text/x-wiki ptgqsjgps9o9un3gk5tup66s8qbtrm0 2449314 2449313 2026-08-18T19:32:55Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1168.eqiad.wmnet with reason: host reimage 2449314 wikitext text/x-wiki d8ptmub1zna6ouhxfvubivkmq2u7833 2449315 2449314 2026-08-18T19:32:58Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1167.eqiad.wmnet with reason: host reimage 2449315 wikitext text/x-wiki rgwsuff5hlqyf0jchaqsuwmpsayrzqa 2449316 2449315 2026-08-18T19:34:06Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1203.eqiad.wmnet with reason: host reimage 2449316 wikitext text/x-wiki fumeuooh2wjjn0ag6s8091sopkt6gkq 2449317 2449316 2026-08-18T19:37:48Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1166.eqiad.wmnet with reason: host reimage 2449317 wikitext text/x-wiki bxz6kxey92q7bmv0mi3nb1ag6ms7sza 2449318 2449317 2026-08-18T19:41:21Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1167.eqiad.wmnet with reason: host reimage 2449318 wikitext text/x-wiki pghcy1lgw6hl9esxk8vbqdz9tmk4372 2449319 2449318 2026-08-18T19:42:33Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1326907|Revert "Disable redis lock manager on testwiki"]] 2449319 wikitext text/x-wiki 9ijw4fhxb7ivem79sfseiske2nbub0c 2449320 2449319 2026-08-18T19:44:42Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1326907|Revert "Disable redis lock manager on testwiki"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449320 wikitext text/x-wiki ehnzfrm0txngexwz358kdvx1handbxt 2449321 2449320 2026-08-18T19:46:08Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2449321 wikitext text/x-wiki 5j8y6xnx5wcc6dj95b6z9t7xyig8dy8 2449322 2449321 2026-08-18T19:47:12Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1204.eqiad.wmnet with reason: host reimage 2449322 wikitext text/x-wiki lr1bti5umzmu8op05sy54ehx8fowd44 2449323 2449322 2026-08-18T19:50:35Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1168.eqiad.wmnet with reason: host reimage 2449323 wikitext text/x-wiki 8bl68zgvqyzdhtk09v7alu85q579mi5 2449325 2449323 2026-08-18T19:53:39Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326907|Revert "Disable redis lock manager on testwiki"]] (duration: 11m 05s) 2449325 wikitext text/x-wiki tqu15h968on4bw264aq8pu7lq4w8iwq 2449326 2449325 2026-08-18T19:54:59Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1203.eqiad.wmnet with OS bookworm 2449326 wikitext text/x-wiki atzn2uvszylaktxengyx2wjjjp37dy5 2449327 2449326 2026-08-18T20:02:33Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1166.eqiad.wmnet with OS bookworm 2449327 wikitext text/x-wiki tudot4ttuo8de2unud2ur7ppfaxpsby 2449328 2449327 2026-08-18T20:05:46Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1167.eqiad.wmnet with OS bookworm 2449328 wikitext text/x-wiki du3x5u8bmve098qq5diitt2z4xdxoba 2449329 2449328 2026-08-18T20:08:13Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1204.eqiad.wmnet with OS bookworm 2449329 wikitext text/x-wiki q7q0duhslyb75cvrn3xiwmakw2rbwqy 2449330 2449329 2026-08-18T20:08:52Z Stashbot 7414 zabe: zabe@deploy1003:~$ mwscript extensions/WikimediaMaintenance/maintenance/fixFileRevisionArchiveNameDrift.php enwiki # T428406 2449330 wikitext text/x-wiki 2lgw1hm7v05om202yjhfdzzc4fxqllw 2449332 2449330 2026-08-18T20:14:23Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1168.eqiad.wmnet with OS bookworm 2449332 wikitext text/x-wiki 4yurhrtvseycnszdrfvjvoyck58fixe 2449333 2449332 2026-08-18T20:20:45Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2449333 wikitext text/x-wiki axz3boy9m96c5se8nnl1lnoo13029js 2449334 2449333 2026-08-18T20:23:19Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2449334 wikitext text/x-wiki 4w9bqna2crx33vc11aissosrdoz3w1g 2449335 2449334 2026-08-18T20:23:40Z Stashbot 7414 kemayo@deploy1003: Started scap sync-world: Backport for [[gerrit:1326870|LLMSuggestionsEditCheck: don't over-cache the description (T428641)]], [[gerrit:1326871|LLMSuggestionsEditCheck: don't over-cache the description (T428641)]] 2449335 wikitext text/x-wiki d7udn7spox62edsu4u7r5kqwldjvt8x 2449336 2449335 2026-08-18T20:25:47Z Stashbot 7414 kemayo@deploy1003: kemayo: Backport for [[gerrit:1326870|LLMSuggestionsEditCheck: don't over-cache the description (T428641)]], [[gerrit:1326871|LLMSuggestionsEditCheck: don't over-cache the description (T428641)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449336 wikitext text/x-wiki 5k7oyop597jaxgq9d8ii5ss1kmho0ff 2449337 2449336 2026-08-18T20:26:45Z Stashbot 7414 ryankemper: `an-launcher1003` confirmed the host is flapping because of memory thrash. chasing down the source of the thrash 2449337 wikitext text/x-wiki qo7uykt2se8hojdl2pktlf8nx2zemit 2449338 2449337 2026-08-18T20:26:47Z Stashbot 7414 kemayo@deploy1003: kemayo: Continuing with deployment 2449338 wikitext text/x-wiki lwaolxxwq3mrka0c07gex32gebmqbwm 2449339 2449338 2026-08-18T20:28:57Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2449339 wikitext text/x-wiki jpjybwjysh7ija7x9t3s08hq5fmjfyd 2449340 2449339 2026-08-18T20:31:15Z Stashbot 7414 kemayo@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326870|LLMSuggestionsEditCheck: don't over-cache the description (T428641)]], [[gerrit:1326871|LLMSuggestionsEditCheck: don't over-cache the description (T428641)]] (duration: 07m 35s) 2449340 wikitext text/x-wiki gsnn1qhea612cfp30fpb6l01fmlfbch 2449341 2449340 2026-08-18T20:31:18Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2449341 wikitext text/x-wiki fhepovwylxy385dvuzs5stv0kh7unuk 2449342 2449341 2026-08-18T20:33:46Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2449342 wikitext text/x-wiki emmeww5urldz4no9xqqp3koyrhscb7p 2449343 2449342 2026-08-18T20:35:50Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2449343 wikitext text/x-wiki gwnioscvc4fu5acecmx6h965um4t61f 2449346 2449343 2026-08-18T20:48:37Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1326896|ext.math.mathjax: Call MathJax.typeset() from `wikipage.content` hook (T434469 T419356 T422077)]], [[gerrit:1326897|ext.math.mathjax: Call MathJax.typeset() from `wikipage.content` hook (T434469 T419356 T422077)]] 2449346 wikitext text/x-wiki t1zgdchpajmrj825zk889kmszgg1m4a 2449347 2449346 2026-08-18T20:49:39Z Stashbot 7414 ryankemper: `an-launcher1003` terminated process group `666809` (`rest_backfill_phase1.sh`) ~20 mins ago with `sudo kill -TERM -- -666809` after its local spark driver (`--driver-memory 64g`) repeatedly exhausted memory on the 32 GB VM and caused SSH to intermittently flap; host recovered to 27 GB available memory 2449347 wikitext text/x-wiki cvca9baysu4vm0tymiz5loqhmw4l5ru 2449348 2449347 2026-08-18T20:50:39Z Stashbot 7414 krinkle@deploy1003: krinkle: Backport for [[gerrit:1326896|ext.math.mathjax: Call MathJax.typeset() from `wikipage.content` hook (T434469 T419356 T422077)]], [[gerrit:1326897|ext.math.mathjax: Call MathJax.typeset() from `wikipage.content` hook (T434469 T419356 T422077)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449348 wikitext text/x-wiki omehzm6h3ixqc5ia1flwhkscs25otbb 2449349 2449348 2026-08-18T20:58:10Z Stashbot 7414 krinkle@deploy1003: krinkle: Continuing with deployment 2449349 wikitext text/x-wiki c34bmlw4fut47gpevlvvxmuocn9azrb 2449350 2449349 2026-08-18T21:02:36Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326896|ext.math.mathjax: Call MathJax.typeset() from `wikipage.content` hook (T434469 T419356 T422077)]], [[gerrit:1326897|ext.math.mathjax: Call MathJax.typeset() from `wikipage.content` hook (T434469 T419356 T422077)]] (duration: 13m 56s) 2449350 wikitext text/x-wiki a3tdoy5znm0b4antkumcafwicus79qx 2449352 2449350 2026-08-18T21:24:26Z Stashbot 7414 ryankemper@cumin2003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on an-worker1204.eqiad.wmnet with reason: 1204 datanode repair T434494 2449352 wikitext text/x-wiki 51tk1bh99g2sn1mjfkdkb7719mwmf80 2449353 2449352 2026-08-18T21:31:05Z Stashbot 7414 caro@deploy1003: Started scap sync-world: Backport for [[gerrit:1326925|Exclude NPOV-type LLM-generated suggestions (T435253)]], [[gerrit:1326926|Exclude NPOV-type LLM-generated suggestions (T435253)]], [[gerrit:1326929|LLMSuggestionsEditCheck: final comparison should also have the object-replacements]], [[gerrit:1326928|LLMSuggestionsEditCheck: final comparison should also have the object-replacements]] 2449353 wikitext text/x-wiki o4ifhmdxqp2qr5e68i1wxy8wvln1f15 2449354 2449353 2026-08-18T21:33:09Z Stashbot 7414 caro@deploy1003: caro: Backport for [[gerrit:1326925|Exclude NPOV-type LLM-generated suggestions (T435253)]], [[gerrit:1326926|Exclude NPOV-type LLM-generated suggestions (T435253)]], [[gerrit:1326929|LLMSuggestionsEditCheck: final comparison should also have the object-replacements]], [[gerrit:1326928|LLMSuggestionsEditCheck: final comparison should also have the object-replacements]] synced to the testservers (see h 2449354 wikitext text/x-wiki swa0dhfnxusc9hqg45ic432f4v8j86f 2449355 2449354 2026-08-18T21:34:16Z Stashbot 7414 caro@deploy1003: caro: Continuing with deployment 2449355 wikitext text/x-wiki bclq23faq07t44my5el8bjng78wbfo2 2449356 2449355 2026-08-18T21:38:32Z Stashbot 7414 caro@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326925|Exclude NPOV-type LLM-generated suggestions (T435253)]], [[gerrit:1326926|Exclude NPOV-type LLM-generated suggestions (T435253)]], [[gerrit:1326929|LLMSuggestionsEditCheck: final comparison should also have the object-replacements]], [[gerrit:1326928|LLMSuggestionsEditCheck: final comparison should also have the object-replacements]] (duration: 0 2449356 wikitext text/x-wiki thq8fx1mfnepdjzutmvdhy0g2x0cjta 2449357 2449356 2026-08-18T21:54:57Z Stashbot 7414 sbassett: Deployed security fix for T435234 (wmf.15) 2449357 wikitext text/x-wiki nzzf0vzq5obrjvjm99wlhgend8n7agj 2449358 2449357 2026-08-18T22:04:15Z Stashbot 7414 sbassett: Deployed security fix for T435234 (wmf.16) 2449358 wikitext text/x-wiki mzrdmbd1r9c2n58u8jric4dlt1n367n 2449359 2449358 2026-08-18T22:10:58Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1326923|Enable redis lock manager on s6 (T366938)]] 2449359 wikitext text/x-wiki muih4mvkhqho75vm2mbsa3r5w8ltsd6 2449360 2449359 2026-08-18T22:13:02Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1326923|Enable redis lock manager on s6 (T366938)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449360 wikitext text/x-wiki 0s828p94dgrvhzcellzyx8hvk257bmn 2449361 2449360 2026-08-18T22:18:31Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2449361 wikitext text/x-wiki 885xceftik83yo5qt1lybjpzyaph1x7 2449362 2449361 2026-08-18T22:22:51Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326923|Enable redis lock manager on s6 (T366938)]] (duration: 11m 52s) 2449362 wikitext text/x-wiki hi2me80bz8nxgfwece9ck8p36mu1fk8 2449363 2449362 2026-08-18T22:33:14Z Stashbot 7414 ryankemper@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-worker1194.eqiad.wmnet with OS bookworm 2449363 wikitext text/x-wiki 1kwbbo3os72ult6c29kp57phw2pa7ha 2449364 2449363 2026-08-18T22:35:37Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1170.eqiad.wmnet with OS bookworm 2449364 wikitext text/x-wiki 97gdwppjt170iqg7noxigkir021eht9 2449365 2449364 2026-08-18T22:35:44Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1171.eqiad.wmnet with OS bookworm 2449365 wikitext text/x-wiki qethudm7770d3xgnej24uhy7quyet7e 2449366 2449365 2026-08-18T22:35:50Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1186.eqiad.wmnet with OS bookworm 2449366 wikitext text/x-wiki ntd2r4wkfn9dmp1gvllt3usfqggg6uo 2449367 2449366 2026-08-18T22:35:57Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1205.eqiad.wmnet with OS bookworm 2449367 wikitext text/x-wiki 0zyb6rjjja7mfyymwhsq7ytog4x5lma 2449368 2449367 2026-08-18T22:36:09Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1206.eqiad.wmnet with OS bookworm 2449368 wikitext text/x-wiki nhmbuo47e4q73yrn3u5xx43jdco3y54 2449372 2449368 2026-08-18T22:50:26Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1206.eqiad.wmnet with reason: host reimage 2449372 wikitext text/x-wiki ep0st71p5vlvgqnvvqlbtep4w7uibp5 2449374 2449372 2026-08-18T22:51:12Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1170.eqiad.wmnet with reason: host reimage 2449374 wikitext text/x-wiki 3vmm1m85kxmsfhbkxmk39hs4kpcambn 2449375 2449374 2026-08-18T22:51:15Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1171.eqiad.wmnet with reason: host reimage 2449375 wikitext text/x-wiki 058npduyj7wdbcqk67fk6ooy6j4g28n 2449376 2449375 2026-08-18T22:53:40Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1205.eqiad.wmnet with reason: host reimage 2449376 wikitext text/x-wiki 7olg12i7iau9zsk5kc1nyf7uvvff6jj 2449377 2449376 2026-08-18T22:54:30Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1206.eqiad.wmnet with reason: host reimage 2449377 wikitext text/x-wiki nai1t682dngniu9g9mmjxang2dknzye 2449378 2449377 2026-08-18T22:57:10Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1171.eqiad.wmnet with reason: host reimage 2449378 wikitext text/x-wiki b7f3rf5ryv8h9pr5nwy0ces11sc91u8 2449379 2449378 2026-08-18T23:00:24Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1205.eqiad.wmnet with reason: host reimage 2449379 wikitext text/x-wiki 3ufhon5p6lkio7cqdlnrjttd813pqiv 2449380 2449379 2026-08-18T23:05:02Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1170.eqiad.wmnet with reason: host reimage 2449380 wikitext text/x-wiki ap7jfe6y888nwvlv6oq2ox4vj2x1v1m 2449381 2449380 2026-08-18T23:15:15Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1206.eqiad.wmnet with OS bookworm 2449381 wikitext text/x-wiki elmmxxneic2rq4diwhslj87tfh9ki8i 2449382 2449381 2026-08-18T23:20:43Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1171.eqiad.wmnet with OS bookworm 2449382 wikitext text/x-wiki 0nisbrxnfhtj6y4ngymceczg46f11oo 2449383 2449382 2026-08-18T23:21:12Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1205.eqiad.wmnet with OS bookworm 2449383 wikitext text/x-wiki ii8keod7303zfk9rkfeenalikiqfn78 2449384 2449383 2026-08-18T23:27:53Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1170.eqiad.wmnet with OS bookworm 2449384 wikitext text/x-wiki g78nvumdr2kr79g3rdflvtn69c2pnbh 2449385 2449384 2026-08-18T23:29:41Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1326944|Retire filebackend lock manager in favour of the default one (T366938)]] 2449385 wikitext text/x-wiki 58x3anl9a6iil6410vx0njcb9sx25mv 2449386 2449385 2026-08-18T23:31:45Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1326944|Retire filebackend lock manager in favour of the default one (T366938)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449386 wikitext text/x-wiki 0lzk4awj0f0r0zewm9k3onazgjfnpi4 2449387 2449386 2026-08-18T23:34:20Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2449387 wikitext text/x-wiki oln1bvbt9dkfgstugpb8rfav4ijo4bb 2449388 2449387 2026-08-18T23:38:36Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326944|Retire filebackend lock manager in favour of the default one (T366938)]] (duration: 08m 55s) 2449388 wikitext text/x-wiki 32xm58vbx3malbt13qzddg9oheun8f6 2449389 2449388 2026-08-18T23:45:55Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1264845|Remove unused/redundant wgMFNoindexPages=true setting (T255458)]] 2449389 wikitext text/x-wiki ri6crls1cmnid074uyqu6cwr2yqkppu 2449390 2449389 2026-08-18T23:48:02Z Stashbot 7414 krinkle@deploy1003: krinkle: Backport for [[gerrit:1264845|Remove unused/redundant wgMFNoindexPages=true setting (T255458)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449390 wikitext text/x-wiki 0usk7vkzl5s0zjin9s5a9ke3tfscqvu 2449391 2449390 2026-08-18T23:51:24Z Stashbot 7414 krinkle@deploy1003: krinkle: Continuing with deployment 2449391 wikitext text/x-wiki dwp0r4i7qzb0lzxg534tfgfd8ijkpoq 2449393 2449391 2026-08-18T23:55:39Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1264845|Remove unused/redundant wgMFNoindexPages=true setting (T255458)]] (duration: 09m 42s) 2449393 wikitext text/x-wiki 3vn661vm22knlpigki8c788qre6x1id 2449404 2449393 2026-08-19T02:00:43Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2449404 wikitext text/x-wiki 0lji5oqicjjhpk0hv81ypibu65582xo 2449405 2449404 2026-08-19T02:07:32Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 48s) 2449405 wikitext text/x-wiki 4io79mnbmc8amtyqn2lol236fjehuyb 2449408 2449405 2026-08-19T02:18:06Z Stashbot 7414 denisse@cumin1003: START - Cookbook sre.mysql.depool depool db2207: Depooling replica 2449408 wikitext text/x-wiki 56q1mc2ggiu5cfcma7cnpyw1c7je9ba 2449409 2449408 2026-08-19T02:18:09Z Stashbot 7414 denisse@cumin1003: END (FAIL) - Cookbook sre.mysql.depool (exit_code=99) depool db2207: Depooling replica 2449409 wikitext text/x-wiki o680nf6meequjs285ya0lczbp5l0s2s 2449410 2449409 2026-08-19T02:29:50Z Stashbot 7414 ryankemper@cumin2003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-worker1186.eqiad.wmnet with OS bookworm 2449410 wikitext text/x-wiki 0kcv8wx9t59cpxfbvc24rrop92jcezy 2449411 2449410 2026-08-19T02:32:10Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1186.eqiad.wmnet with OS bookworm 2449411 wikitext text/x-wiki 7bf6yf1t1q4d4y5sdn6x1g14or2jbrh 2449413 2449411 2026-08-19T02:39:43Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 28 hosts with reason: Primary switchover s2 T435270 2449413 wikitext text/x-wiki msgdj207z846j7bzcapude7vfe4b110 2449414 2449413 2026-08-19T02:39:55Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Set db2204 with weight 0 T435270', diff saved to https://phabricator.wikimedia.org/P96179 and previous config saved to /var/cache/conftool/dbconfig/20260819-023951-marostegui.json 2449414 wikitext text/x-wiki iv4idjevo5p8by3z9hb4k001ll8d62u 2449416 2449414 2026-08-19T02:43:42Z Stashbot 7414 marostegui: Starting s2 codfw failover from db2207 to db2204 - T435270 2449416 wikitext text/x-wiki 32kb2lh9ytzpxwk1lbcrcnlvjxwmg02 2449417 2449416 2026-08-19T02:44:08Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Promote db2204 to s2 primary T435270', diff saved to https://phabricator.wikimedia.org/P96180 and previous config saved to /var/cache/conftool/dbconfig/20260819-024403-marostegui.json 2449417 wikitext text/x-wiki kquc2n9nf7aag3lqede54ryaeai3s7v 2449418 2449417 2026-08-19T02:46:31Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Depool db2207 T435270', diff saved to https://phabricator.wikimedia.org/P96181 and previous config saved to /var/cache/conftool/dbconfig/20260819-024627-marostegui.json 2449418 wikitext text/x-wiki cd77xqtyttxv8zdv134n2zcoqgou69c 2449419 2449418 2026-08-19T02:46:36Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1186.eqiad.wmnet with reason: host reimage 2449419 wikitext text/x-wiki d34wekoxjhqpqn1ukk2f5l5hq7wbq8o 2449422 2449419 2026-08-19T02:54:33Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1186.eqiad.wmnet with reason: host reimage 2449422 wikitext text/x-wiki ppqagtmvdksxl9qx0soph5ca6s8rpvg 2449423 2449422 2026-08-19T03:16:41Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1186.eqiad.wmnet with OS bookworm 2449423 wikitext text/x-wiki 6iaaatug8zigcah1mjygljik45jut9c 2449427 2449423 2026-08-19T05:51:04Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db2209: db2209 repool 2449427 wikitext text/x-wiki phyoe7liofddlyaic7iaqwr3k6x4jcb 2449429 2449427 2026-08-19T05:59:15Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2048.codfw.wmnet 2449429 wikitext text/x-wiki jbrfgb1wf7nkwszc7zm4yy4a94ar3wv 2449430 2449429 2026-08-19T06:03:56Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2048.codfw.wmnet 2449430 wikitext text/x-wiki h9wd5bqurjgb58s58llkozis8akhphh 2449431 2449430 2026-08-19T06:06:27Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1284 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96184 and previous config saved to /var/cache/conftool/dbconfig/20260819-060621-marostegui.json 2449431 wikitext text/x-wiki t9hnump4nrvb5ud8r8hrzllas2me87l 2449432 2449431 2026-08-19T06:06:47Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1284: Pool back 2449432 wikitext text/x-wiki hrl7tml3sdo88pznjnx6as460na6jht 2449433 2449432 2026-08-19T06:09:28Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2048.codfw.wmnet 2449433 wikitext text/x-wiki c7h4jyw1p8ib6mpisa1zhwjf7zhki2y 2449434 2449433 2026-08-19T06:10:08Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2048.codfw.wmnet 2449434 wikitext text/x-wiki 2sd1arwcl2lb4vkpq1d7qpeggqmdmxx 2449435 2449434 2026-08-19T06:10:10Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host zuul1002.eqiad.wmnet 2449435 wikitext text/x-wiki 9lb4lyimst4new6rkcqukczvahr88zl 2449436 2449435 2026-08-19T06:14:16Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host zuul1002.eqiad.wmnet 2449436 wikitext text/x-wiki pm39tzmuyo6e7q5n4jvset8u5plzkvs 2449437 2449436 2026-08-19T06:14:27Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host zuul1003.eqiad.wmnet 2449437 wikitext text/x-wiki 6eqid0ucr2zjljx75nkvkh3q83z0esd 2449438 2449437 2026-08-19T06:17:48Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1284 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96186 and previous config saved to /var/cache/conftool/dbconfig/20260819-061743-marostegui.json 2449438 wikitext text/x-wiki euvwkgsq6593yw6svjiusvpammef31k 2449439 2449438 2026-08-19T06:18:13Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1280: Pool back 2449439 wikitext text/x-wiki j5fd2gvsssmysbp39xggbvoestvwmls 2449440 2449439 2026-08-19T06:18:17Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host zuul1003.eqiad.wmnet 2449440 wikitext text/x-wiki n9n6amipxfyfrxrk5js9zos2dky9w8c 2449441 2449440 2026-08-19T06:18:42Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host zuul1001.eqiad.wmnet 2449441 wikitext text/x-wiki de2jf6uzcsk08oa4a6fwalwnl3sdam8 2449442 2449441 2026-08-19T06:22:51Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host zuul1001.eqiad.wmnet 2449442 wikitext text/x-wiki arm7e8jyok1eh7ybl03rxjbdfurw0gz 2449443 2449442 2026-08-19T06:26:52Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host bast3007.wikimedia.org 2449443 wikitext text/x-wiki mxf8hf7ln2p8jqss1vbbu6pefveipu3 2449444 2449443 2026-08-19T06:28:19Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1277 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96190 and previous config saved to /var/cache/conftool/dbconfig/20260819-062815-marostegui.json 2449444 wikitext text/x-wiki 0qklr1x29ies801i3n0zxxlt720tca4 2449445 2449444 2026-08-19T06:29:48Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1277: Pool back 2449445 wikitext text/x-wiki 6f2i1f1gg2iuahnlsbxtp4l8o09sd4m 2449446 2449445 2026-08-19T06:33:00Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast3007.wikimedia.org 2449446 wikitext text/x-wiki hjw4nw8tki0fozt0t7n0t74qk1ic262 2449447 2449446 2026-08-19T06:33:47Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host aux-k8s-etcd1004.eqiad.wmnet 2449447 wikitext text/x-wiki b08beo21j7npx4zrzcuz2s1175g764n 2449448 2449447 2026-08-19T06:36:15Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2209: db2209 repool 2449448 wikitext text/x-wiki jsc0uemaef9ykdiqnjzbt4utpmnpk51 2449449 2449448 2026-08-19T06:37:35Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host aux-k8s-etcd1004.eqiad.wmnet 2449449 wikitext text/x-wiki onx4n7vf1doz3uypbk789w7myxue59y 2449450 2449449 2026-08-19T06:41:12Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host aux-k8s-etcd1005.eqiad.wmnet 2449450 wikitext text/x-wiki oq7fhr5zb84bbwk2y4sca2ocsa9v1s0 2449451 2449450 2026-08-19T06:45:13Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host aux-k8s-etcd1005.eqiad.wmnet 2449451 wikitext text/x-wiki gagcv59g1136cjdf767we8qqr27g6ea 2449452 2449451 2026-08-19T06:49:17Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.clone of db1222.eqiad.wmnet onto db1276.eqiad.wmnet 2449452 wikitext text/x-wiki o0hikrdcyk0qnx51557m2592x6agan7 2449453 2449452 2026-08-19T06:49:21Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1222: Depool db1222.eqiad.wmnet to then clone it to db1276.eqiad.wmnet - marostegui@cumin1003 2449453 wikitext text/x-wiki 63j0jgoard55buckn1nkmwhws2ygnff 2449454 2449453 2026-08-19T06:49:36Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1222: Depool db1222.eqiad.wmnet to then clone it to db1276.eqiad.wmnet - marostegui@cumin1003 2449454 wikitext text/x-wiki rh9476exe3j9l7ggg1otlpr0cs86wp5 2449455 2449454 2026-08-19T06:51:56Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1284: Pool back 2449455 wikitext text/x-wiki f3r3db8072fqe1usnyz22zjl6wn2f76 2449456 2449455 2026-08-19T06:54:23Z Stashbot 7414 moritzm: installing php8.2 security updates 2449456 wikitext text/x-wiki if21h1qvbt1vb9id2x52g6fea8hkhja 2449457 2449456 2026-08-19T06:55:39Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host netmon2002.wikimedia.org 2449457 wikitext text/x-wiki 387iv2va3sfwjmj7jxb7gtesbqinckl 2449458 2449457 2026-08-19T07:01:21Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netmon2002.wikimedia.org 2449458 wikitext text/x-wiki hgxkbt9yb7aryttd4l8irfboxiemeqc 2449459 2449458 2026-08-19T07:03:22Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1280: Pool back 2449459 wikitext text/x-wiki g8r2ihse8ztp16f97jdmiiw9e8hkdzw 2449460 2449459 2026-08-19T07:07:46Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host netmon1003.wikimedia.org 2449460 wikitext text/x-wiki fvp6dypt6qpvgwcbspjlfhq4hbymin6 2449461 2449460 2026-08-19T07:14:44Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netmon1003.wikimedia.org 2449461 wikitext text/x-wiki 2qvzvk5dm4nsqfr5okjzhycrb937w4y 2449462 2449461 2026-08-19T07:14:57Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1277: Pool back 2449462 wikitext text/x-wiki 7ya6f68xjm5noarpf3gyle7cfyulq1y 2449463 2449462 2026-08-19T07:17:20Z Stashbot 7414 moritzm: installing imagemagick security updates 2449463 wikitext text/x-wiki qw9qldjisb2kpueiwkwajkiw4nzrhlu 2449465 2449463 2026-08-19T07:35:22Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.swift.convert-disks for host ms-be1064 2449465 wikitext text/x-wiki dnrwn5ola4bdbavpgnumjvsui0ospo4 2449466 2449465 2026-08-19T07:36:58Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.swift.convert-disks for host ms-be1068 2449466 wikitext text/x-wiki ptxuyflo2hryd62qtewrc4ywpc66gkq 2449467 2449466 2026-08-19T07:42:55Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1222: Pool db1222.eqiad.wmnet in after cloning 2449467 wikitext text/x-wiki g787pbh1x9hx7pnp99ld7ru2sopoy4z 2449468 2449467 2026-08-19T07:45:44Z Stashbot 7414 moritzm: extend the disk of ldap-rw2001 by 80G T331699 2449468 wikitext text/x-wiki arwovt9yqnegturizf7cnsmhrg956sj 2449469 2449468 2026-08-19T07:48:07Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1055.eqiad.wmnet with OS trixie 2449469 wikitext text/x-wiki njwh4jj1zwzfronodkhap2nxet6qgfx 2449470 2449469 2026-08-19T07:58:48Z Stashbot 7414 arnaudb@dns1006: START - running authdns-update 2449470 wikitext text/x-wiki qjsubpw1eij3mmtd5vohx0sn060x83j 2449471 2449470 2026-08-19T07:59:50Z Stashbot 7414 arnaudb@dns1006: START - running authdns-update 2449471 wikitext text/x-wiki im9tywjs0er5o3hck2hjzke6nxawssj 2449472 2449471 2026-08-19T08:01:38Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1172.eqiad.wmnet with OS bookworm 2449472 wikitext text/x-wiki l2i03hn11no0linrmx2izmu3ielk2pl 2449473 2449472 2026-08-19T08:01:42Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1173.eqiad.wmnet with OS bookworm 2449473 wikitext text/x-wiki kci0mmclostp8r6wriezbhkjt49oh04 2449474 2449473 2026-08-19T08:01:46Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1174.eqiad.wmnet with OS bookworm 2449474 wikitext text/x-wiki hcd8mnxjkeulvroc16zskek310e8dtp 2449475 2449474 2026-08-19T08:02:01Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1207.eqiad.wmnet with OS bookworm 2449475 wikitext text/x-wiki qd74f7dpb2c6d6e9yptagp7qxgk7e6b 2449476 2449475 2026-08-19T08:02:04Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1208.eqiad.wmnet with OS bookworm 2449476 wikitext text/x-wiki oub4lamjvu9u5flzkf4j1cglnh8xec0 2449477 2449476 2026-08-19T08:02:07Z Stashbot 7414 arnaudb@dns1006: END - running authdns-update 2449477 wikitext text/x-wiki dya1mj2rs2t6oe6d78oojkvkr2bsgob 2449478 2449477 2026-08-19T08:03:41Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1055.eqiad.wmnet with reason: host reimage 2449478 wikitext text/x-wiki 8wbmeiavouae8v0wg9mclhd8q7b30pv 2449479 2449478 2026-08-19T08:07:21Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on pfw1-codfw with reason: upgrade pfw1a-codfw and pfw1b-codfw pair 2449479 wikitext text/x-wiki cnh5tlz7rwpylqiqvp8ehjs8gszpy7v 2449480 2449479 2026-08-19T08:09:32Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1055.eqiad.wmnet with reason: host reimage 2449480 wikitext text/x-wiki jmhfgadfxdcmly7ykoemyi8q0oa33k3 2449481 2449480 2026-08-19T08:10:32Z Stashbot 7414 cmooney@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on cr[1-2]-codfw with reason: upgrade pfw1a-codfw and pfw1b-codfw pair 2449481 wikitext text/x-wiki algfkqpvwiidv3cejt06l7zn4gnu3ta 2449482 2449481 2026-08-19T08:13:02Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db2902.codfw.wmnet with OS trixie 2449482 wikitext text/x-wiki hgc26a1nt3hu8ulnwmkj6joquz8h8t4 2449483 2449482 2026-08-19T08:13:05Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host db2902.codfw.wmnet 2449483 wikitext text/x-wiki od531y1nhlf5jlylwuzpz2swr509k8i 2449484 2449483 2026-08-19T08:14:07Z Stashbot 7414 aklapper@deploy1003: rebuilt and synchronized wikiversions files: group1 to 1.47.0-wmf.16 refs T430835 2449484 wikitext text/x-wiki 7oahmbv3bjlrbbzm8qvxtrkea5q8fvs 2449485 2449484 2026-08-19T08:14:40Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.reimage for host db2902.codfw.wmnet with OS trixie 2449485 wikitext text/x-wiki 9dq3z6qzr0egkrtudpedp5wfp0c676n 2449486 2449485 2026-08-19T08:15:17Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1207.eqiad.wmnet with reason: host reimage 2449486 wikitext text/x-wiki m5dp57535m4yjpvs97uci3s88z8gj51 2449487 2449486 2026-08-19T08:15:19Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1208.eqiad.wmnet with reason: host reimage 2449487 wikitext text/x-wiki m58raxfp4zi98u4zfes44lvfz6y4ztz 2449488 2449487 2026-08-19T08:15:41Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host mc-misc2002.codfw.wmnet 2449488 wikitext text/x-wiki 4kwpi19wn14jizyw8844sokbq8x88sy 2449489 2449488 2026-08-19T08:17:03Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1172.eqiad.wmnet with reason: host reimage 2449489 wikitext text/x-wiki 31oz0he37kchg29uxqaqi4kuc1kjfpz 2449490 2449489 2026-08-19T08:17:14Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1173.eqiad.wmnet with reason: host reimage 2449490 wikitext text/x-wiki 4wjdhn6onu9vlmpknaealq8n13quf5l 2449491 2449490 2026-08-19T08:17:41Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1174.eqiad.wmnet with reason: host reimage 2449491 wikitext text/x-wiki 5aovtfbkusvbrew4c9n9fsv3babbkyu 2449492 2449491 2026-08-19T08:18:49Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1207.eqiad.wmnet with reason: host reimage 2449492 wikitext text/x-wiki 1641wa1eega5y11ru99m6cj5viaw5im 2449493 2449492 2026-08-19T08:20:25Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.swift.convert-disks for host ms-be1065 2449493 wikitext text/x-wiki 245gv5qy6zism507i2is4dwjcjkbf7r 2449494 2449493 2026-08-19T08:20:46Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host ms-be1064.eqiad.wmnet with OS trixie 2449494 wikitext text/x-wiki 1tkydjmdcfku3hmlzwf1lx3cf93wxv5 2449495 2449494 2026-08-19T08:22:22Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1172.eqiad.wmnet with reason: host reimage 2449495 wikitext text/x-wiki 0kc07ld99bdglf4ommvf7gw2quzqhww 2449496 2449495 2026-08-19T08:23:50Z Stashbot 7414 topranks: reboot pfw1-codfw firewall pair to upgrade JunOS T434865 2449496 wikitext text/x-wiki q5vq2b7r6ycto2om4u28g9xvx7g7cnt 2449497 2449496 2026-08-19T08:24:45Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-misc2002.codfw.wmnet 2449497 wikitext text/x-wiki hr86kp93823450eynpl5g3kcyja1h5b 2449498 2449497 2026-08-19T08:25:05Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - filippo@cumin1003" 2449498 wikitext text/x-wiki todrtpunes5l0h9ock49mpc4cpvy5xu 2449499 2449498 2026-08-19T08:26:25Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1174.eqiad.wmnet with reason: host reimage 2449499 wikitext text/x-wiki eum433kftpoy5w321bpw0wxj7bfj1uw 2449500 2449499 2026-08-19T08:27:41Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - filippo@cumin1003" 2449500 wikitext text/x-wiki qtu1fvbnyuuahryaruekz09zzkydzsz 2449501 2449500 2026-08-19T08:27:44Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1055.eqiad.wmnet with OS trixie 2449501 wikitext text/x-wiki mfcnzcsep78uiogtz1qbuyivi7eraht 2449502 2449501 2026-08-19T08:28:06Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on db2902.codfw.wmnet with reason: host reimage 2449502 wikitext text/x-wiki nct5sar2c47x4yp3ugsfryh375zqags 2449503 2449502 2026-08-19T08:28:09Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1222: Pool db1222.eqiad.wmnet in after cloning 2449503 wikitext text/x-wiki ngrog2eh8lq3s01xpg2qi10b0973mje 2449504 2449503 2026-08-19T08:28:18Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1222.eqiad.wmnet onto db1276.eqiad.wmnet 2449504 wikitext text/x-wiki 951l3iicizpghtdn0rmks60c9hrtmrq 2449505 2449504 2026-08-19T08:29:36Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1056.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2449505 wikitext text/x-wiki sj2c4dpxpxetjeoe4dbajudwtga5odc 2449506 2449505 2026-08-19T08:31:45Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1057.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2449506 wikitext text/x-wiki bpaalikq3glh15ar48d43t39mx1bhj2 2449507 2449506 2026-08-19T08:34:09Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1208.eqiad.wmnet with reason: host reimage 2449507 wikitext text/x-wiki khc4w6dtbwq31y017f4r0doktt2r9e9 2449508 2449507 2026-08-19T08:34:56Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host ms-be1068.eqiad.wmnet with OS trixie 2449508 wikitext text/x-wiki ivnfw4r63loxm2zzmyyedyuldtcnhhu 2449509 2449508 2026-08-19T08:37:59Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be1064.eqiad.wmnet with reason: host reimage 2449509 wikitext text/x-wiki jlb9w8dbzw6j5c6wlkp73shc537tby8 2449510 2449509 2026-08-19T08:38:17Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2902.codfw.wmnet with reason: host reimage 2449510 wikitext text/x-wiki 08md8ma3xmcpvklhy73wd82zdq4e5ek 2449511 2449510 2026-08-19T08:39:14Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1207.eqiad.wmnet with OS bookworm 2449511 wikitext text/x-wiki mc7dy2x083evrggabiai59vkqghyhd3 2449512 2449511 2026-08-19T08:39:32Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1057.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2449512 wikitext text/x-wiki m5bdz4t16eor7t0vwyasa3ofrkn2b2d 2449513 2449512 2026-08-19T08:40:47Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1057.eqiad.wmnet with OS trixie 2449513 wikitext text/x-wiki 27yelc9u7s1g0otxmopoarfya85f8gp 2449514 2449513 2026-08-19T08:41:49Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be1064.eqiad.wmnet with reason: host reimage 2449514 wikitext text/x-wiki 25cmfyf3yvu9xrgf51thirtx44zoj9v 2449515 2449514 2026-08-19T08:45:35Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1173.eqiad.wmnet with reason: host reimage 2449515 wikitext text/x-wiki e9bxctyyzdinxx7koaae3r0wbhtyryf 2449516 2449515 2026-08-19T08:46:03Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1172.eqiad.wmnet with OS bookworm 2449516 wikitext text/x-wiki 0q8zvqbq0xma4726j4ukff12jjnf8pq 2449517 2449516 2026-08-19T08:50:32Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1174.eqiad.wmnet with OS bookworm 2449517 wikitext text/x-wiki d74j2j880hhgthjk5z27ifw9ynt6fr0 2449518 2449517 2026-08-19T08:52:14Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be1068.eqiad.wmnet with reason: host reimage 2449518 wikitext text/x-wiki d76lb31n8sm7gec6z98z1gm2thk6xqj 2449519 2449518 2026-08-19T08:53:11Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2902.codfw.wmnet with OS trixie 2449519 wikitext text/x-wiki f8d9rpfmqm4b4z8debkeo2hpbcymjd6 2449520 2449519 2026-08-19T08:54:14Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1208.eqiad.wmnet with OS bookworm 2449520 wikitext text/x-wiki 273z9g4v1w7kq4ku5bj64vd2qugc79u 2449521 2449520 2026-08-19T08:56:06Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1057.eqiad.wmnet with reason: host reimage 2449521 wikitext text/x-wiki phx5tbfjdqgyfok3ecaexje84b2y0br 2449522 2449521 2026-08-19T08:58:46Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be1068.eqiad.wmnet with reason: host reimage 2449522 wikitext text/x-wiki iz4ltesfp03a7nuw06rsiqizcgt0eu8 2449523 2449522 2026-08-19T09:01:29Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1056.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2449523 wikitext text/x-wiki ihoe12jcpedzztofctn2dpxn8zxwhgv 2449524 2449523 2026-08-19T09:02:52Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1057.eqiad.wmnet with reason: host reimage 2449524 wikitext text/x-wiki 9rarb06v6duqipmf1tzu7lvloddt4hk 2449525 2449524 2026-08-19T09:03:06Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1056.eqiad.wmnet with OS trixie 2449525 wikitext text/x-wiki of0g5ecv5zocv5tu1jadk5tvjaq6rd9 2449526 2449525 2026-08-19T09:03:53Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be1064.eqiad.wmnet with OS trixie 2449526 wikitext text/x-wiki 7vo21zfmm8owsfegmeu8fqbgyzuyoqe 2449527 2449526 2026-08-19T09:03:56Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.swift.convert-disks (exit_code=0) for host ms-be1064 2449527 wikitext text/x-wiki nwyj4sz8ri948na9bzwppnumnf5uiaw 2449528 2449527 2026-08-19T09:06:40Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reimage for host ldap-rw2001.wikimedia.org with OS trixie 2449528 wikitext text/x-wiki 0s4bkwq4pcqjvzj6o1evl9vhbqrmay7 2449529 2449528 2026-08-19T09:07:23Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1173.eqiad.wmnet with OS bookworm 2449529 wikitext text/x-wiki 7bn49n9niyl0atnk46835wek97tqcky 2449530 2449529 2026-08-19T09:07:26Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.mysql.update-replication 2449530 wikitext text/x-wiki 09uikqevc1i1fqkb4op709lq4hcfdfm 2449531 2449530 2026-08-19T09:07:52Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.mysql.update-replication (exit_code=97) 2449531 wikitext text/x-wiki 30cxnfb08ielodkyanb95w6wpwssqvh 2449532 2449531 2026-08-19T09:09:37Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host irc1003.wikimedia.org 2449532 wikitext text/x-wiki 5dwfku6vohj3x42zefsmtbhhszfukqi 2449533 2449532 2026-08-19T09:09:50Z Stashbot 7414 moritzm: installing Postgresql security updates 2449533 wikitext text/x-wiki 3fr750g946d31mze06qmrzj55ct6cwf 2449534 2449533 2026-08-19T09:12:52Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2449534 wikitext text/x-wiki 9dfc0ki9patx4gbf2smp2kirj8upbsp 2449535 2449534 2026-08-19T09:13:18Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.reimage for host ms-be1065.eqiad.wmnet with OS trixie 2449535 wikitext text/x-wiki 8tytl8cunhz0vbgjoi6w20cs1193j5b 2449536 2449535 2026-08-19T09:13:25Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2449536 wikitext text/x-wiki 553x40y8ilyfyw1h38m75fg7y1wgh0e 2449537 2449536 2026-08-19T09:13:27Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host irc1003.wikimedia.org 2449537 wikitext text/x-wiki h0q8hlkrbtq6nhp1rgjxbu1vye8sc2l 2449538 2449537 2026-08-19T09:14:53Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2449538 wikitext text/x-wiki k0bi6fxiggyaymx8rxvmg9jmwb26uw2 2449539 2449538 2026-08-19T09:15:49Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2449539 wikitext text/x-wiki 8l9geez7j55vzn5ry1amp1rgzmutk2z 2449540 2449539 2026-08-19T09:18:26Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - filippo@cumin1003" 2449540 wikitext text/x-wiki 79s51n6ifiuyyauo214hp8r3sc9z669 2449541 2449540 2026-08-19T09:18:29Z Stashbot 7414 filippo@cumin1003: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - filippo@cumin1003" 2449541 wikitext text/x-wiki 8p09y2gw83ph8e3p8zbiedx6je4oxmm 2449542 2449541 2026-08-19T09:18:31Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1057.eqiad.wmnet with OS trixie 2449542 wikitext text/x-wiki 1qp9bf5x6z2ztefmh2kub0esknyt9ms 2449543 2449542 2026-08-19T09:18:37Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1056.eqiad.wmnet with reason: host reimage 2449543 wikitext text/x-wiki 6eplw8h0vhnzmhv3uieqshjupzqhltb 2449544 2449543 2026-08-19T09:19:25Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "cloudvirt1057 - filippo@cumin1003" 2449544 wikitext text/x-wiki g361oi7g8g3txl6clj842s5vphsg64b 2449545 2449544 2026-08-19T09:19:39Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "cloudvirt1057 - filippo@cumin1003" 2449545 wikitext text/x-wiki szq0kwf1mahgydka4yw6aojk3ktia51 2449546 2449545 2026-08-19T09:20:04Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1054.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2449546 wikitext text/x-wiki 836su1e88wssd7wgjbpjffd5yra1t64 2449547 2449546 2026-08-19T09:20:41Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be1068.eqiad.wmnet with OS trixie 2449547 wikitext text/x-wiki cthz3odr1urabj1i7k24yyrtlu5gw6n 2449548 2449547 2026-08-19T09:20:43Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.swift.convert-disks (exit_code=0) for host ms-be1068 2449548 wikitext text/x-wiki erkisf526n2yt4wkj9ytsgos9fd9jm0 2449549 2449548 2026-08-19T09:23:53Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host netboxdb2003.codfw.wmnet 2449549 wikitext text/x-wiki klt2ou91232hv6hki0r0of6rhge80ne 2449550 2449549 2026-08-19T09:24:01Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1056.eqiad.wmnet with reason: host reimage 2449550 wikitext text/x-wiki 3fu0crtqxup5hc3tth015qqvlipvj3a 2449551 2449550 2026-08-19T09:26:26Z Stashbot 7414 arnaudb@dns1006: START - running authdns-update 2449551 wikitext text/x-wiki n97pfoeniqed9ex3dulfoiswbn4eio6 2449552 2449551 2026-08-19T09:27:39Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netboxdb2003.codfw.wmnet 2449552 wikitext text/x-wiki qki4bgu9wnx3s49qmufkbctpxr8f2zd 2449553 2449552 2026-08-19T09:28:01Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on ldap-rw2001.wikimedia.org with reason: host reimage 2449553 wikitext text/x-wiki iz6i6kfyw26qe4no55gqq58s71fogxq 2449554 2449553 2026-08-19T09:28:36Z Stashbot 7414 arnaudb@dns1006: END - running authdns-update 2449554 wikitext text/x-wiki p2a9xftzo2wrgxay90zguxtgsebs3yb 2449555 2449554 2026-08-19T09:30:51Z Stashbot 7414 mvernon@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be1065.eqiad.wmnet with reason: host reimage 2449555 wikitext text/x-wiki rlqkg2txav39zge9wndc9k69kkidclq 2449556 2449555 2026-08-19T09:31:12Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host netboxdb1003.eqiad.wmnet 2449556 wikitext text/x-wiki rasvt5kksd2ujk921rja14dttxmzamu 2449557 2449556 2026-08-19T09:33:20Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ldap-rw2001.wikimedia.org with reason: host reimage 2449557 wikitext text/x-wiki dz567itque41q88ids8j6bcrb4s5jvg 2449558 2449557 2026-08-19T09:34:52Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netboxdb1003.eqiad.wmnet 2449558 wikitext text/x-wiki nnwjcnynqed2e8yt5asjkxhh959194c 2449559 2449558 2026-08-19T09:36:22Z Stashbot 7414 filippo@cumin1003: END (ERROR) - Cookbook sre.hosts.provision (exit_code=97) for host cloudvirt1054.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2449559 wikitext text/x-wiki nye1gv0xgafokd53obhdy53n97iq0n6 2449560 2449559 2026-08-19T09:36:44Z Stashbot 7414 topranks: make HE transport circuits from magru live 2449560 wikitext text/x-wiki 1fagb32co7basihkv7o3mqwsivi59rg 2449561 2449560 2026-08-19T09:36:58Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be1065.eqiad.wmnet with reason: host reimage 2449561 wikitext text/x-wiki m3mlu8yaluvq472fpxu92hlaw7z3ssh 2449562 2449561 2026-08-19T09:39:36Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - filippo@cumin1003" 2449562 wikitext text/x-wiki 18z392rbw4n2j7xp65g4czkhwxc9d0e 2449563 2449562 2026-08-19T09:44:04Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - filippo@cumin1003" 2449563 wikitext text/x-wiki q1afmm2sk9257xmawpf8klrmd87tun9 2449564 2449563 2026-08-19T09:44:08Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1056.eqiad.wmnet with OS trixie 2449564 wikitext text/x-wiki rlhw0crlos6prlw9mibak90qgnkm155 2449565 2449564 2026-08-19T09:46:25Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1054.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2449565 wikitext text/x-wiki fyeasa9r724gak697t5meignd0g6ksc 2449566 2449565 2026-08-19T09:51:29Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.netbox.restart-reboot rolling reboot on A:netbox 2449566 wikitext text/x-wiki 3ail9k1c566g5qlhbu721m65k69nltn 2449567 2449566 2026-08-19T09:51:32Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.dns.wipe-cache netbox.discovery.wmnet. on all recursors 2449567 wikitext text/x-wiki tn2ywpye4k1volwrylssl4p4a0cb8s6 2449568 2449567 2026-08-19T09:51:36Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) netbox.discovery.wmnet. on all recursors 2449568 wikitext text/x-wiki pp4dqzpr5qe5taqr4r267h5pbfzzqap 2449569 2449568 2026-08-19T09:51:57Z Stashbot 7414 filippo@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudvirt1054.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2449569 wikitext text/x-wiki ky32lwo6hk25rqnj9et4aoggjo0fs8b 2449570 2449569 2026-08-19T09:52:07Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ldap-rw2001.wikimedia.org with OS trixie 2449570 wikitext text/x-wiki 6k67ov63hg1lqdf2h24451s8w2cp515 2449572 2449570 2026-08-19T09:55:51Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.dns.wipe-cache netbox.discovery.wmnet. on all recursors 2449572 wikitext text/x-wiki ammcbhlc57whnrfs6ayu48kxjvmo1tx 2449573 2449572 2026-08-19T09:55:55Z Stashbot 7414 mpostoronca@deploy1003: Started scap sync-world: Backport for [[gerrit:1321579|Register the mediawiki.wikimedia_antiabuse.content_policy_score stream (T432848)]] 2449573 wikitext text/x-wiki dlcbxjw2a1eugwk78p6xh0l13owss2q 2449574 2449573 2026-08-19T09:55:58Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) netbox.discovery.wmnet. on all recursors 2449574 wikitext text/x-wiki 36fndsuc857ps2h3sri9ulk0u93q0q9 2449575 2449574 2026-08-19T09:58:09Z Stashbot 7414 mpostoronca@deploy1003: mpostoronca: Backport for [[gerrit:1321579|Register the mediawiki.wikimedia_antiabuse.content_policy_score stream (T432848)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449575 wikitext text/x-wiki cmsf7l75do66i6gi3j1itva499kw86i 2449576 2449575 2026-08-19T09:59:02Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be1065.eqiad.wmnet with OS trixie 2449576 wikitext text/x-wiki fsljzjn2qgjzy4gsqzt7byoz7vzxe94 2449577 2449576 2026-08-19T09:59:05Z Stashbot 7414 mvernon@cumin2003: END (PASS) - Cookbook sre.swift.convert-disks (exit_code=0) for host ms-be1065 2449577 wikitext text/x-wiki 6r0995ljvrr3el1dc8bcdwk107o6lbd 2449578 2449577 2026-08-19T09:59:17Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1169.eqiad.wmnet with OS bookworm 2449578 wikitext text/x-wiki e9u98f1esxuvdocp0v63zbzygdxgon8 2449579 2449578 2026-08-19T09:59:20Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1185.eqiad.wmnet with OS bookworm 2449579 wikitext text/x-wiki 1mbtigncjpwlealjytpq1iv4n3rl7mk 2449580 2449579 2026-08-19T10:00:18Z Stashbot 7414 mpostoronca@deploy1003: mpostoronca: Continuing with deployment 2449580 wikitext text/x-wiki 0jsuxizgg5tx4ucddgixhvph4wy9vzc 2449581 2449580 2026-08-19T10:02:43Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2449581 wikitext text/x-wiki 2jhgif8d524suupke7hvviafksudjz4 2449582 2449581 2026-08-19T10:03:20Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2449582 wikitext text/x-wiki httwyx03ab8ctq4oy7jsxz6d33i3qxz 2449583 2449582 2026-08-19T10:03:53Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2449583 wikitext text/x-wiki 7tckdmb8xwvs81xvtob3jrqdt2gqc8t 2449584 2449583 2026-08-19T10:04:46Z Stashbot 7414 mpostoronca@deploy1003: Finished scap sync-world: Backport for [[gerrit:1321579|Register the mediawiki.wikimedia_antiabuse.content_policy_score stream (T432848)]] (duration: 08m 53s) 2449584 wikitext text/x-wiki 8ih9fplivxwr70kprqsowptfy76wo3c 2449585 2449584 2026-08-19T10:04:52Z Stashbot 7414 atsuko@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2449585 wikitext text/x-wiki 2m7u0gjhees7kwacqkg46wi62i2z7nh 2449586 2449585 2026-08-19T10:06:44Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host stat1008.eqiad.wmnet with OS bookworm 2449586 wikitext text/x-wiki 8f8c2q3oidd12dst19z6xi5hwc3ixe6 2449587 2449586 2026-08-19T10:11:43Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.netbox.restart-reboot (exit_code=0) rolling reboot on A:netbox 2449587 wikitext text/x-wiki 83lojv4e8lhs2xqc2gkfm9tib8klfdy 2449588 2449587 2026-08-19T10:14:25Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1185.eqiad.wmnet with reason: host reimage 2449588 wikitext text/x-wiki ps2ar895y85nd9ygwtf1frzr3y04eif 2449589 2449588 2026-08-19T10:14:54Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1169.eqiad.wmnet with reason: host reimage 2449589 wikitext text/x-wiki oxjv0yaengdopcc0l44trezjdeeblrr 2449593 2449589 2026-08-19T10:19:38Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1185.eqiad.wmnet with reason: host reimage 2449593 wikitext text/x-wiki rp4uow4t6mvcijujzp3394bgdq35zbj 2449595 2449593 2026-08-19T10:23:12Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1169.eqiad.wmnet with reason: host reimage 2449595 wikitext text/x-wiki g598naxa2xyh4z5ehzsco1dhv4twptr 2449602 2449595 2026-08-19T10:42:09Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1185.eqiad.wmnet with OS bookworm 2449602 wikitext text/x-wiki pdzhd5ub2qflomi6evcajgyvvrgdwif 2449603 2449602 2026-08-19T10:45:44Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1169.eqiad.wmnet with OS bookworm 2449603 wikitext text/x-wiki n0um9dn28ivuv8bkig047fa8c6p36su 2449604 2449603 2026-08-19T10:56:07Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host cumin2002.codfw.wmnet 2449604 wikitext text/x-wiki pha06oyfm13rer5suwshugfspfsq1ji 2449605 2449604 2026-08-19T11:01:47Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cumin2002.codfw.wmnet 2449605 wikitext text/x-wiki shogds1ta86gefq0u18gwl5mdqcj6cn 2449606 2449605 2026-08-19T11:01:54Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . 2449606 wikitext text/x-wiki q4us8pb44koyu5oqg4p0aouiyrnnbb2 2449607 2449606 2026-08-19T11:06:30Z Stashbot 7414 moritzm: upgrading the new trixie URL downloaders to Squid 7.6 T427282 2449607 wikitext text/x-wiki 1iebbbao7wdcpevhut9se8rp4tbygsy 2449608 2449607 2026-08-19T11:07:27Z Stashbot 7414 btullis@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host stat1008.eqiad.wmnet with OS bookworm 2449608 wikitext text/x-wiki tmzsr8837014hpr3whlx7vzeey10nbu 2449609 2449608 2026-08-19T11:07:36Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host maps-test2001.codfw.wmnet 2449609 wikitext text/x-wiki rp8zqxti1djkz1eup128ji2wa4ucg07 2449610 2449609 2026-08-19T11:08:12Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-cron: apply 2449610 wikitext text/x-wiki 2oji79n6cm53f0cpbdj89yjq67s8s79 2449611 2449610 2026-08-19T11:08:19Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-cron: apply 2449611 wikitext text/x-wiki 8tikeuzwv089pc2umby4hwy3ox825ov 2449612 2449611 2026-08-19T11:08:22Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/mw-cron: apply 2449612 wikitext text/x-wiki 9zibxlbdcfth8o04opiegytbb8hp6x1 2449613 2449612 2026-08-19T11:08:27Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-cron: apply 2449613 wikitext text/x-wiki frqormfnxeq00j20b69yn6awlan0ufy 2449614 2449613 2026-08-19T11:10:21Z Stashbot 7414 kartik@deploy1003: helmfile [staging] START helmfile.d/services/machinetranslation: apply 2449614 wikitext text/x-wiki 318s7shskdafvgq11jqwodwyw7p6phz 2449615 2449614 2026-08-19T11:12:15Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] START helmfile.d/services/citoid: apply 2449615 wikitext text/x-wiki 2q43wbtxlc2qbvkwu5g77rqda8yiu4x 2449616 2449615 2026-08-19T11:12:32Z Stashbot 7414 mvolz@deploy1003: helmfile [staging] DONE helmfile.d/services/citoid: apply 2449616 wikitext text/x-wiki kie10abdc73avexzjx9gzlukk1rmerh 2449617 2449616 2026-08-19T11:13:19Z Stashbot 7414 kartik@deploy1003: helmfile [staging] DONE helmfile.d/services/machinetranslation: apply 2449617 wikitext text/x-wiki fyi6rkr1xgegt2w0f9loj2hhg69sgf0 2449618 2449617 2026-08-19T11:17:04Z Stashbot 7414 kartik@deploy1003: helmfile [codfw] START helmfile.d/services/machinetranslation: apply 2449618 wikitext text/x-wiki h5r85vcqyks182dw07lm1vu1cuzt2u5 2449619 2449618 2026-08-19T11:18:27Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps-test2001.codfw.wmnet 2449619 wikitext text/x-wiki rmqo6buwtazeph0fo42nu2jywdptyxy 2449620 2449619 2026-08-19T11:18:36Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host stat1008.eqiad.wmnet with OS bookworm 2449620 wikitext text/x-wiki sb9nxb6vyitkf4l96jwqan2rmfso9f6 2449621 2449620 2026-08-19T11:20:06Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1327084|Revert "Migrate database access to virtual domains" (T435305)]] 2449621 wikitext text/x-wiki 4kb3ldectdysowrlttfot6a6459jzze 2449622 2449621 2026-08-19T11:21:02Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] START helmfile.d/services/citoid: apply 2449622 wikitext text/x-wiki 8p4pfs85nr5tpyyyqw787hnag21a98u 2449623 2449622 2026-08-19T11:21:31Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] DONE helmfile.d/services/citoid: apply 2449623 wikitext text/x-wiki m1uy6n0w608r6jnz2v9w408s0jcxfev 2449624 2449623 2026-08-19T11:21:35Z Stashbot 7414 kartik@deploy1003: helmfile [codfw] DONE helmfile.d/services/machinetranslation: apply 2449624 wikitext text/x-wiki l0luiefdkxg3t9z5o048vkqc66shzcr 2449625 2449624 2026-08-19T11:22:12Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1327084|Revert "Migrate database access to virtual domains" (T435305)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2449625 wikitext text/x-wiki 8s9vjqjl3zpllb41x7a4qcl1news56g 2449627 2449625 2026-08-19T11:23:24Z Stashbot 7414 kartik@deploy1003: helmfile [eqiad] START helmfile.d/services/machinetranslation: apply 2449627 wikitext text/x-wiki mh4uqttq72s5bpf4lr6fjeke7s5o1qi 2449628 2449627 2026-08-19T11:23:28Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] START helmfile.d/services/citoid: apply 2449628 wikitext text/x-wiki 8o9pg0s5a8ct0ck8cela6pox231el44 2449629 2449628 2026-08-19T11:23:57Z Stashbot 7414 mvolz@deploy1003: helmfile [codfw] DONE helmfile.d/services/citoid: apply 2449629 wikitext text/x-wiki 2pl4mnvc3dt63lfweqbpgt9z951bjlv 2449630 2449629 2026-08-19T11:24:09Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] START helmfile.d/services/citoid: apply 2449630 wikitext text/x-wiki ea0jyeswgh3utx3fi8p1qwhnifk9az5 2449631 2449630 2026-08-19T11:24:41Z Stashbot 7414 mvolz@deploy1003: helmfile [eqiad] DONE helmfile.d/services/citoid: apply 2449631 wikitext text/x-wiki ltjpbk6hxfg7x4llsieu8fc08d1uopf 2449632 2449631 2026-08-19T11:26:49Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2449632 wikitext text/x-wiki 6asssflplevbe7ormb9fz7l185n2r38 2449633 2449632 2026-08-19T11:28:47Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . 2449633 wikitext text/x-wiki kq97m95kc3ykl0ms5r0ig6hc4hkr1yj 2449634 2449633 2026-08-19T11:29:01Z Stashbot 7414 kartik@deploy1003: helmfile [eqiad] DONE helmfile.d/services/machinetranslation: apply 2449634 wikitext text/x-wiki s9l6bpopxpg7uvu6icp67n8exj7l6ct 2449635 2449634 2026-08-19T11:29:23Z Stashbot 7414 kart_: Updated MinT to 2026-06-04-131507-production (T321316) 2449635 wikitext text/x-wiki 6avu68t7nxxosiokre6fw59yhe8gwfz 2449636 2449635 2026-08-19T11:29:49Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . 2449636 wikitext text/x-wiki t7rd7xkp784ri5qn2zqwyd18j5kfz7o 2449637 2449636 2026-08-19T11:30:00Z Stashbot 7414 gkyziridis@deploy1003: helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . 2449637 wikitext text/x-wiki hrn0r58u9ee8vnodadi72ptoqg79pbc 2449639 2449637 2026-08-19T11:31:08Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1327084|Revert "Migrate database access to virtual domains" (T435305)]] (duration: 11m 02s) 2449639 wikitext text/x-wiki 4vgqukmpkwfn49civ8q47t46fa5nxds 2449642 2449639 2026-08-19T11:44:01Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on stat1008.eqiad.wmnet with reason: host reimage 2449642 wikitext text/x-wiki drba3um46jte2zqwglo08dql2kumhjm 2449643 2449642 2026-08-19T11:48:19Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on stat1008.eqiad.wmnet with reason: host reimage 2449643 wikitext text/x-wiki lndjfnq1qhyqvqdwm8fruqlg6sp3p6u 2449644 2449643 2026-08-19T11:49:40Z Stashbot 7414 moritzm: installing kerberos security updates 2449644 wikitext text/x-wiki gi9aohpa303hqqg75kwioer5ac0a7gq 2449646 2449644 2026-08-19T11:58:34Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host krb2002.codfw.wmnet 2449646 wikitext text/x-wiki 7o7sjiml7p5a7eatd63gol87g52biv0 0 9393 2449369 2382510 2026-08-18T22:43:34Z ADenisse-WMF 31462 Add instructions to reboot logstash hosts. 2449369 wikitext text/x-wiki kkhzhp80a5i92ebqfvxpqf2k3y8kx6g 2449370 2449369 2026-08-18T22:46:21Z ADenisse-WMF 31462 /* Rebooting hosts */ 2449370 wikitext text/x-wiki pd5y29jucw8wc3tnlq6j5r61oykblyo 2449371 2449370 2026-08-18T22:46:41Z ADenisse-WMF 31462 /* Rebooting hosts */ 2449371 wikitext text/x-wiki jtxsh94zpfwhlhim36rnzzlcn83tluk 2449373 2449371 2026-08-18T22:51:02Z ADenisse-WMF 31462 /* Rebooting hosts */ 2449373 wikitext text/x-wiki e1s9iqkl9dq6x9kk0rwca9h4ar282oe 2449392 2449373 2026-08-18T23:54:33Z CWhite (WMF) 12916 /* Rebooting storage nodes: */ 2449392 wikitext text/x-wiki 0fxh694cy3piqjls6g3mh2xpvkw7amp 2449394 2449392 2026-08-18T23:55:59Z CWhite (WMF) 12916 /* Steps with commands: */ 2449394 wikitext text/x-wiki fjta4bbniknhfzqfv10xk80c3wg49s3 2449395 2449394 2026-08-18T23:57:07Z CWhite (WMF) 12916 /* Rebooting collector nodes: */ 2449395 wikitext text/x-wiki 9lz0045jspapgz2p78k27q7s6dm4arw 2449396 2449395 2026-08-18T23:57:48Z CWhite (WMF) 12916 /* Steps with commands: */ 2449396 wikitext text/x-wiki cc08yg0977aqonp5l0d6lt3ggey5wxi 0 12858 2449270 2396748 2026-08-18T17:54:02Z AKlapper (WMF) 421 Remove an 11y old outdated example 2449270 wikitext text/x-wiki f82wonsp921s8i452ht913qw675kebv 0 17290 2449257 2448844 2026-08-18T17:31:32Z Stashbot 7414 dancy: rm /var/log/apache2/*.gz on deployment-mediawiki14.deployment-prep.eqiad1.wikimedia.cloud to free up ~6GB. 2449257 wikitext text/x-wiki g3432s814w4yu0rls42wohuw136c4ln 2449258 2449257 2026-08-18T17:32:28Z Stashbot 7414 dancy: Rebooting deployment-mediawiki14.deployment-prep.eqiad1.wikimedia.cloud for good measure 2449258 wikitext text/x-wiki lo4gjob0cp15iwnfc1pnfdvbldu468n 0 26586 2449300 2448670 2026-08-18T19:05:15Z ADenisse-WMF 31462 Add incident-response-health-metrics. 2449300 wikitext text/x-wiki 1mwyhslmxu732b84wi5elzfc6rne894 498 30942 2449260 2448679 2026-08-18T17:36:31Z Stashbot 7414 andrewbogott: changing ssd write cache to 'off' and 'write-through' on all drives in cloudcephosd1046, restarting osd services T429387 2449260 wikitext text/x-wiki s7aojn2xj39dkif2kv2yl0ltv8qhsbs 2449295 2449260 2026-08-18T18:49:20Z Stashbot 7414 andrewbogott: rebooting cloudcephosd1046 to pick up (or reset?) drive caching changes 2449295 wikitext text/x-wiki 0yb6hj235lckyxh9tnbvuwd68q4p0ah 2449406 2449295 2026-08-19T02:09:59Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.set_cluster_in_maintenance 2449406 wikitext text/x-wiki n3ohmzcqfb3suyfaon51s8q5y802mkw 2449407 2449406 2026-08-19T02:10:02Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.set_cluster_in_maintenance (exit_code=0) 2449407 wikitext text/x-wiki 7gm7agqr75i08ctqogqw331ms4eqnl2 2449420 2449407 2026-08-19T02:51:09Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.unset_cluster_maintenance 2449420 wikitext text/x-wiki 7pjwd6ruukus4giwgnkqj3isth8a9m5 2449421 2449420 2026-08-19T02:51:11Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.unset_cluster_maintenance (exit_code=0) 2449421 wikitext text/x-wiki 849bfc3t47aado59j6d983ivdfdv5gn 0 441095 2449119 2423222 2026-08-18T13:47:34Z CMooney (WMF) 24288 /* AlertManager alerts */ 2449119 wikitext text/x-wiki pz9a795op8lg3f13kovuhil84wwgm5y 0 441820 2449064 2444683 2026-08-18T12:41:09Z Lucas Werkmeister (WMDE) 10037 /* GUI deployment general notes */ add notes for testing during deployment using port forwarding (h/t [[User:KSoučková-WMF]] and [[User:CGoubert-WMF]]) 2449064 wikitext text/x-wiki t2a8i011a17n0wwgtiox4sqv5l4wpnk 0 443657 2449199 2253278 2026-08-18T15:49:39Z Hoo man 76 /* Each update */ ++ 2449199 wikitext text/x-wiki 8duwbmtpmrtml94gu19987irfe7h8u1 498 443713 2449215 2445592 2026-08-18T16:09:53Z Stashbot 7414 wmbot~valeriobozzolan@tools-bastion-15: deletionbot: released latest changes - up to commit https://gitlab.wikimedia.org/valeriobozzolan/itwiki-deletions-info-bot/-/commit/59c1c01f8d0ac3ac12862070c19600bd0f1a7599 2449215 wikitext text/x-wiki mjwv5ra1wyy5bztgj6rrrqdt2ker6l0 2449228 2449215 2026-08-18T16:42:54Z Stashbot 7414 wmbot~valeriobozzolan@tools-bastion-15: deletionbot: released latest changes - up to commit https://gitlab.wikimedia.org/valeriobozzolan/itwiki-deletions-info-bot/-/commit/8399b56bbe918fee3b9cd1074e41ea05ad2b8bc7 2449228 wikitext text/x-wiki 7m04vcnyvnpvpocsjjwlzimatv7c9x7 498 443800 2449024 2448522 2026-08-18T12:13:12Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed 42e38be327 via push-to-deploy 2449024 wikitext text/x-wiki dksh0298ko3fr99rj5cl0f0sxu6dw6z 3 444547 2449425 2331432 2026-08-19T04:33:37Z Sean.hoyland 37387 /* Question about block */ new section 2449425 wikitext text/x-wiki nt647c52yl6mkmpffloeenwl4r729cd 2449426 2449425 2026-08-19T04:37:06Z Sean.hoyland 37387 /* Question about block */ link 2449426 wikitext text/x-wiki f76475ozeio2u7wme5ztmmk8up9td2x 0 447352 2449082 2423111 2026-08-18T13:03:37Z LToscano (WMF) 7191 Add backend section plus migration info 2449082 wikitext text/x-wiki on5en1dcgnqtqlrd7u3g4x0yp3t9oxr 0 447668 2449290 2237020 2026-08-18T18:24:42Z Saper 135 /* Usual errors */ Puppet CRL expired 2449290 wikitext text/x-wiki stktqnm8zdnp54cqliwmb2bfcsbrc7w 0 448441 2449274 2072744 2026-08-18T17:59:18Z AKlapper (WMF) 421 remove outdated deployment server name 2449274 wikitext text/x-wiki rh5wgpowx156pvecvey0zpp4fo3iyro 2449275 2449274 2026-08-18T17:59:46Z AKlapper (WMF) 421 use consistent wording 2449275 wikitext text/x-wiki 2wo2b210jm4bnmnw0w4ovfpai0ydcaa 0 448962 2449571 2443533 2026-08-19T09:55:50Z JMeybohm (WMF) 16709 /* Generate diff to upstream */ remove crd special handling 2449571 wikitext text/x-wiki gm8x7osvg4amkq24hvh43584p9wsrkx 2449591 2449571 2026-08-19T10:18:00Z JMeybohm (WMF) 16709 /* Update deployment-charts chart */ 2449591 wikitext text/x-wiki a7sen84uoiiadsnrqag463zgyftjrcz 0 449160 2449398 2448671 2026-08-19T00:01:22Z Dexbot 30554 Bot: Updating the report 2449398 wikitext text/x-wiki qg86kcie69fqcmuzdbnuiv0v6vj9tqp 2449415 2449398 2026-08-19T02:40:48Z Dexbot 30554 Bot: Updating the report 2449415 wikitext text/x-wiki ple2i99gjdbw9tnjwftpl4gns4by7qv 3 453751 2449399 2433595 2026-08-19T00:09:44Z JJPMaster 21716 JJPMaster moved page [[User talk:DreamRimmer]] to [[User talk:Max]]: Automatically moved page while renaming the user "[[Special:CentralAuth/DreamRimmer|DreamRimmer]]" to "[[Special:CentralAuth/Max|Max]]" 2283159 wikitext text/x-wiki j0liykodw36ucspl6296m9bkrbvpcru 0 454129 2449294 2443688 2026-08-18T18:45:50Z Daimona Eaytoy 11462 /* Configure TestKitchen extension */ The address was wrong (it's 172, not 127), but just suggest adding the extra host and checking the firewall instead 2449294 wikitext text/x-wiki qcwapr9pkfc3wbmwr9hw8udch06i4by 2 454410 2449401 2433594 2026-08-19T00:09:44Z JJPMaster 21716 JJPMaster moved page [[User:DreamRimmer]] to [[User:Max]]: Automatically moved page while renaming the user "[[Special:CentralAuth/DreamRimmer|DreamRimmer]]" to "[[Special:CentralAuth/Max|Max]]" 2169426 wikitext text/x-wiki i2po8upj05sc6yj7hblbp3cieyunwk6 498 454435 2449034 2444682 2026-08-18T12:23:00Z Stashbot 7414 taavi: make webservice deployment be managed by components-api 2449034 wikitext text/x-wiki 2invvrzuppkddfjaajnu38gfnoj16se 0 455989 2449331 2438104 2026-08-18T20:10:58Z CMing (WMF) 23277 /* URL query parameter */ 2449331 wikitext text/x-wiki geaoh47gtkwviprje1g76duc0pww0by 0 456526 2449638 2433441 2026-08-19T11:30:32Z CGoubert-WMF 32775 /* Troubleshooting */ Add dirty awk loop one liners to delete or reschedule all failed jobs 2449638 wikitext text/x-wiki dckm8wwyc9vi4lbc4tfwu2hnugy7w89 2449640 2449638 2026-08-19T11:35:03Z CGoubert-WMF 32775 /* How do I restart all of my team's failed jobs? */ Use xargs instead of loop for deletion 2449640 wikitext text/x-wiki mgtwwyfeyfnbmckxm81nk2lmglt5wet 0 459041 2449205 2394086 2026-08-18T16:02:24Z SFrench-WMF 37362 /* Preparation for WMF */ Fix example link in #5 (title case consistency mapping) 2449205 wikitext text/x-wiki mr0dmx6nbvhkmaza3r2u8b76n3q9b0a 2449218 2449205 2026-08-18T16:25:25Z SFrench-WMF 37362 /* Post-rollout */ Consolidate title-case mapping cleanup into one step (two steps reads as if they can be split up) 2449218 wikitext text/x-wiki 6jouhx7i8hh1imh7eq0qzfqy7td3bit 0 459699 2449202 2448721 2026-08-18T15:54:45Z Stashbot 7414 ejegg: payments-wiki upgraded from 3e339f19 to 07519479 2449202 wikitext text/x-wiki iw63clw0iqq9ubldhbsk4865weq9qfh 2449324 2449202 2026-08-18T19:53:09Z Stashbot 7414 ejegg: payments-wiki upgraded from 07519479 to c059b80a 2449324 wikitext text/x-wiki dj3g028b3ex3b18ei1fi8oa5d2j1970 2449351 2449324 2026-08-18T21:07:05Z Stashbot 7414 eileen: civicrm upgraded from 49047af1 to 08ca9d6c 2449351 wikitext text/x-wiki 5o3qsld64ye30pnwzq0yjb7emuuaps4 2449403 2449351 2026-08-19T00:25:44Z Stashbot 7414 eileen: SmashPig upgraded from 89909858 to 0e3e59b4 2449403 wikitext text/x-wiki fr62fqarhgwh6vsqwphxgvbrcvoj1wk 2449412 2449403 2026-08-19T02:33:42Z Stashbot 7414 eileen: SmashPig upgraded from 0e3e59b4 to 4e9d9df0 2449412 wikitext text/x-wiki erormbkewzkcto00vjoc257h6bxmw45 2449428 2449412 2026-08-19T05:52:33Z Stashbot 7414 eileen: civicrm upgraded from 9049b363 to 4f83dccd 2449428 wikitext text/x-wiki 01qih3byvu9ppz7i98w4jjzgwnu8xit 0 459903 2449641 2446710 2026-08-19T11:42:50Z BJensen-WMF 48942 /* Deploying a new version */ more path specificity 2449641 wikitext text/x-wiki 7wssfgrzbxyt8bjeb2xi8j2g2vd9f53 0 460052 2449168 2448291 2026-08-18T14:42:13Z MPopov (WMF) 5891 /* Experiment assignment query */ Updated query from GB 2449168 wikitext text/x-wiki 5wbq535t616vvmi79mlf3wsivwyueoc 0 460232 2449250 2432414 2026-08-18T17:22:05Z MPopov (WMF) 5891 /* Dig deeper with dimensions */ Documented edit count-based dimensions 2449250 wikitext text/x-wiki e1hftzy1lzhbujfsn2myhl29tf1mmsc 0 460375 2449242 2448689 2026-08-18T17:10:22Z AWesterinen-WMF 54189 /* POINT Decomposition */ 2449242 wikitext text/x-wiki gsqnqo51ahyrk7s3makw0meon7z785u 2449262 2449242 2026-08-18T17:39:38Z AWesterinen-WMF 54189 /* wikibase:around */ 2449262 wikitext text/x-wiki oejnujc4xvrdgqu1mj6h6ju5d5jvs6h 2449297 2449262 2026-08-18T18:51:00Z AWesterinen-WMF 54189 /* wikibase:box */ 2449297 wikitext text/x-wiki 9v8h62v2h6g85zkx27uw7cea7953od1 2449299 2449297 2026-08-18T19:02:15Z AWesterinen-WMF 54189 /* wikibase:box */ 2449299 wikitext text/x-wiki s0kdomu7dssk326vpq96um2c88igtzi 0 460376 2449219 2448726 2026-08-18T16:30:03Z AWesterinen-WMF 54189 /* bd:slice */ 2449219 wikitext text/x-wiki avo1yf5olcfiew9onm2zcqqxc6lgwk4 2449230 2449219 2026-08-18T16:43:36Z AWesterinen-WMF 54189 /* bd:slice */ 2449230 wikitext text/x-wiki gt64tr0e6gapwkxgwvm51nc80usc8g0 0 460515 2449225 2448818 2026-08-18T16:40:49Z Ottomata 88 2449225 wikitext text/x-wiki 4vlew8qs89ghbdrtrhdy57dk333sulz 3 460518 2449397 2444660 2026-08-19T00:01:15Z JJPMaster 21716 JJPMaster moved page [[User talk:Max]] to [[User talk:Max (usurped)]] without leaving a redirect: Automatically moved page while renaming the user "[[Special:CentralAuth/Max|Max]]" to "[[Special:CentralAuth/Max (usurped)|Max (usurped)]]" 2444660 wikitext text/x-wiki ncc3xlywlc5chrnorhtzqdw2a74br39 2 460540 2449203 2448525 2026-08-18T15:57:39Z Ottomata 88 2449203 wikitext text/x-wiki gqdykvakneks3ua1sngbzpr1kmqyyus 3 460562 2449400 2026-08-19T00:09:44Z JJPMaster 21716 JJPMaster moved page [[User talk:DreamRimmer]] to [[User talk:Max]]: Automatically moved page while renaming the user "[[Special:CentralAuth/DreamRimmer|DreamRimmer]]" to "[[Special:CentralAuth/Max|Max]]" 2449400 wikitext text/x-wiki di6dc1hi375rekjkf3vlpmpqeylvbhw 2 460563 2449402 2026-08-19T00:09:44Z JJPMaster 21716 JJPMaster moved page [[User:DreamRimmer]] to [[User:Max]]: Automatically moved page while renaming the user "[[Special:CentralAuth/DreamRimmer|DreamRimmer]]" to "[[Special:CentralAuth/Max|Max]]" 2449402 wikitext text/x-wiki pfiknm5jbdn6swutnnkpq04rmeqt59z 3 460564 2449590 2026-08-19T10:17:13Z StrikerBot 8475 Welcome to Toolforge! 2449590 wikitext text/x-wiki 3pcdefubi91w4uc7oejs1gyns417mqr 3 460565 2449592 2026-08-19T10:18:02Z StrikerBot 8475 Welcome to Toolforge! 2449592 wikitext text/x-wiki lr1xkqs9enuk5b4vny45go4jxd1na9o 3 460566 2449594 2026-08-19T10:21:01Z StrikerBot 8475 Welcome to Toolforge! 2449594 wikitext text/x-wiki iwfs6ppqcv2o20gjuqj7t2jj1045h0v 3 460567 2449626 2026-08-19T11:22:16Z StrikerBot 8475 Welcome to Toolforge! 2449626 wikitext text/x-wiki 98n4d8lza6dltvsqehzb87e5pqsju04