Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.15 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 382 2448907 2390706 2026-08-18T10:06:07Z Stashbot 7414 taavi: rebooting bastion-eqiad1-5 to pick up kernel updates 2448907 wikitext text/x-wiki 0xf5w92pepaa6x6fg0xwi1qs7nphr9u 498 3086 2448153 2447911 2026-08-17T12:26:36Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2448153 wikitext text/x-wiki 4befug3f3feknmwp86uiuqrbmcheojg 2448161 2448153 2026-08-17T12:31:50Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2448161 wikitext text/x-wiki 4z0u3motmmuynkt5jv71cawfdufns5z 2448183 2448161 2026-08-17T12:59:31Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2448183 wikitext text/x-wiki fknyjt84b4qwmy2nruyzk3ahq6whxtm 2448185 2448183 2026-08-17T13:03:19Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2448185 wikitext text/x-wiki ki8ucti7hxe2331va6bu62h2nrcut4b 2448220 2448185 2026-08-17T13:42:53Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logging 2448220 wikitext text/x-wiki c5vx3r3supezntttgaioxm4p7i6so6k 2448226 2448220 2026-08-17T13:48:10Z Stashbot 7414 tlepage@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logging 2448226 wikitext text/x-wiki q9rx8yuz3p2mfl7fpvn1wy5tv3a11iz 2448305 2448226 2026-08-17T14:29:49Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logging 2448305 wikitext text/x-wiki ll0z25ngwh8fxdwfehhgzu5v03ctmdm 2448313 2448305 2026-08-17T14:38:23Z Stashbot 7414 tlepage@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logging 2448313 wikitext text/x-wiki fd1va2onh0qtrh3u0eq5wgl1zgmx4am 2448404 2448313 2026-08-17T15:39:04Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2448404 wikitext text/x-wiki nf2d1s1zw9txkptc051erkt2z44oe9c 2448409 2448404 2026-08-17T15:43:48Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2448409 wikitext text/x-wiki 40t75jofd0ta92gt7qa9et0fhboylgz 2448883 2448409 2026-08-18T09:56:28Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-cli 2448883 wikitext text/x-wiki ffyc2axgqexy67m07y0m3gc54z8p4qh 2448892 2448883 2026-08-18T09:59:43Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-cli 2448892 wikitext text/x-wiki ep1c9a5iszqvti9hygcamremkhqx092 0 4108 2448144 2448038 2026-08-17T12:10:08Z ScheduleDeploymentBot 37566 Add [[gerrit:1325527]] to Monday, August 24 UTC afternoon backport window 2448144 wikitext text/x-wiki 84rpyirf0los56u6m8fjlqni5lyzu2n 2448452 2448144 2026-08-17T16:37:09Z SFrench-WMF 37362 /* {{Deployment_day|date=2026-08-17}} */ UTC late infra window 2448452 wikitext text/x-wiki 0keve61b3iw1g6weiaci42oq87fe8y0 2448578 2448452 2026-08-17T19:37:47Z ScheduleDeploymentBot 37566 Add [[gerrit:1326361]] to Monday, August 17 UTC late backport window 2448578 wikitext text/x-wiki 8edpaa0aouv4j6mad1effr6uhir5l4f 2448587 2448578 2026-08-17T19:59:16Z Superpes15 24252 /* Week of August 17 */ +1 Moving into the right section 2448587 wikitext text/x-wiki g627pc7eeybkcoa990vatdr9usiprnk 2448695 2448587 2026-08-18T02:53:21Z ScheduleDeploymentBot 37566 Add [[gerrit:1326439]] to Tuesday, August 18 UTC late backport window 2448695 wikitext text/x-wiki r7mi45me2leiuoyq9fqaso8u68iaih8 2448696 2448695 2026-08-18T02:54:41Z Krinkle 44 move up 2448696 wikitext text/x-wiki l5lefqfrv926661ncvukrchqrfkglck 2448716 2448696 2026-08-18T03:47:41Z ScheduleDeploymentBot 37566 Add [[gerrit:1326450]] to Tuesday, August 18 UTC morning backport window 2448716 wikitext text/x-wiki fv728geky6c0gav9dlh2bewutpdhud1 2448733 2448716 2026-08-18T05:53:46Z ScheduleDeploymentBot 37566 Add [[gerrit:701016]] to Tuesday, August 18 UTC morning backport window 2448733 wikitext text/x-wiki i5i3ghul7xuvxzbrcvavztzcxccsd3l 2448784 2448733 2026-08-18T08:00:01Z ScheduleDeploymentBot 37566 Add [[gerrit:1326232]] to Tuesday, August 18 UTC afternoon backport window 2448784 wikitext text/x-wiki 0s6hcfmjdxq8tewxnqy4rtvl9mx8ona 2448789 2448784 2026-08-18T08:17:25Z ScheduleDeploymentBot 37566 Add [[gerrit:1326775]] to Tuesday, August 18 UTC afternoon backport window 2448789 wikitext text/x-wiki qkj8btdmaqxmo0iknw1yzavubsonuyp 0 4235 2448315 2404097 2026-08-17T14:38:49Z BKing (WMF) 29017 2448315 wikitext text/x-wiki dj9n0jk53ocbpfdsmkhmomu3hostiod 0 6479 2448312 2443908 2026-08-17T14:37:52Z CGoubert-WMF 32775 Add redis_lock cluster info 2448312 wikitext text/x-wiki fb5bupg9zurshgwbpv65jyee3lxvo7c 498 6726 2448150 2448127 2026-08-17T12:21:47Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2448150 wikitext text/x-wiki 60vsoyci2rg4st6jx4o3j0ejdlfti8m 2448151 2448150 2026-08-17T12:26:05Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2448151 wikitext text/x-wiki rbzv3c6ai02cjz69uwmlgo03q28w4fq 2448165 2448151 2026-08-17T12:34:14Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2448165 wikitext text/x-wiki co6albxka9b41lzdvjvdy7ro9otr5p8 2448170 2448165 2026-08-17T12:37:20Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2448170 wikitext text/x-wiki fvu5lo5mea0r3ogfwr7yk5mi2jkpe56 2448392 2448170 2026-08-17T15:30:42Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api 2448392 wikitext text/x-wiki sr6rowkfubbkejvtsg5euj32nw0ov7o 2448400 2448392 2026-08-17T15:35:35Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api 2448400 wikitext text/x-wiki 1ijl597b6cmdj7f5xpe6jln2j8s1rld 2448865 2448400 2026-08-18T09:46:12Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-cli 2448865 wikitext text/x-wiki mk0nw4315yd5yex9mp17izs3nsg4s2p 2448869 2448865 2026-08-18T09:49:01Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-cli 2448869 wikitext text/x-wiki 57oi07q5d9utbvdlpydlfp6r7er4499 0 7919 2448132 2448131 2026-08-17T12:01:06Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2158-2171].codfw.wmnet 2448132 wikitext text/x-wiki slz4p4tjly3z5vw1anscgq8hwqfx3zr 2448133 2448132 2026-08-17T12:01:10Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2158-2171].codfw.wmnet 2448133 wikitext text/x-wiki 3lm5o3fp8688r225ra45llvun5c4p99 2448134 2448133 2026-08-17T12:01:37Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2172-2179,2184-2189].codfw.wmnet 2448134 wikitext text/x-wiki s5pqluwevufgsdipnu7tgqpavs1j5od 2448135 2448134 2026-08-17T12:02:26Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db2901.codfw.wmnet 2448135 wikitext text/x-wiki 770mfonp0pljs7xjrah6x022lvgnaii 2448136 2448135 2026-08-17T12:02:28Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448136 wikitext text/x-wiki 97rthq2am2rucix6orpoqm7gqitudej 2448137 2448136 2026-08-17T12:02:31Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1027.eqiad.wmnet 2448137 wikitext text/x-wiki 12hs8f12olv9ad82tgpr6eyj980s06x 2448138 2448137 2026-08-17T12:02:40Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.dns.netbox (exit_code=97) 2448138 wikitext text/x-wiki jc5m3hli9ksawmkgl6bq97k3n6pqnld 2448139 2448138 2026-08-17T12:02:44Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1027.eqiad.wmnet 2448139 wikitext text/x-wiki 1i4n2409cnrbeg1crqj8cgoghrbsgt6 2448140 2448139 2026-08-17T12:02:46Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host db2901.codfw.wmnet 2448140 wikitext text/x-wiki o2khm751zdjuk10uu1wlpo23bp1epor 2448141 2448140 2026-08-17T12:02:59Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db2901.codfw.wmnet 2448141 wikitext text/x-wiki 3xqm3p5b280ppwamsy7vlakd7hbwta6 2448142 2448141 2026-08-17T12:03:02Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448142 wikitext text/x-wiki rzaoqzsm2teslno80optlea5hz2fbq1 2448143 2448142 2026-08-17T12:09:47Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1028.eqiad.wmnet 2448143 wikitext text/x-wiki rmc21p8g3e1b9fgkm9empszvrjbeyqg 2448145 2448143 2026-08-17T12:10:33Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2172-2179,2184-2189].codfw.wmnet 2448145 wikitext text/x-wiki kx70sb2o34j3ey656jmq38mlj8t7f7n 2448146 2448145 2026-08-17T12:13:08Z Stashbot 7414 seanleong-wmde@deploy1003: mwscript-k8s job started: foreachwikiindblist wikidataclient extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https # T429955 2448146 wikitext text/x-wiki ehygauf8hk3lfgpq94hikwhad190xqw 2448147 2448146 2026-08-17T12:18:48Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2172-2179,2184-2189].codfw.wmnet 2448147 wikitext text/x-wiki ql0mlulrwyeznzrm814wjfncwmu19i9 2448148 2448147 2026-08-17T12:18:51Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2172-2179,2184-2189].codfw.wmnet 2448148 wikitext text/x-wiki bzqiub86gefzpl6gyuz5jt5o13w5f74 2448149 2448148 2026-08-17T12:19:19Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2190-2203].codfw.wmnet 2448149 wikitext text/x-wiki 4lb39qibh64qgsswfqvukc4igh28nol 2448152 2448149 2026-08-17T12:26:12Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1028.eqiad.wmnet 2448152 wikitext text/x-wiki oa2w86n0u37g8w8v23tkbe92cm22z54 2448154 2448152 2026-08-17T12:27:01Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2448154 wikitext text/x-wiki ay2ydvqataaha4zsu2b9fkqqvdvo2ag 2448155 2448154 2026-08-17T12:27:04Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2190-2203].codfw.wmnet 2448155 wikitext text/x-wiki ojsec0ik3kcusctzf8ikg7akcgnkoeq 2448156 2448155 2026-08-17T12:27:08Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host db2901.codfw.wmnet 2448156 wikitext text/x-wiki e2e821szn6x77kwe1h5rwevk94t8lfd 2448157 2448156 2026-08-17T12:28:00Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.clone of db1170.eqiad.wmnet onto db1284.eqiad.wmnet 2448157 wikitext text/x-wiki 5yjgxhmb9wcb8pjx16de1mi55fm8ruk 2448158 2448157 2026-08-17T12:28:03Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1170: Depool db1170.eqiad.wmnet to then clone it to db1284.eqiad.wmnet - marostegui@cumin1003 2448158 wikitext text/x-wiki baq7h5cj7zr56wf2b7qr9bsct62qx9u 2448159 2448158 2026-08-17T12:28:08Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ldap-maint1001.eqiad.wmnet 2448159 wikitext text/x-wiki k9poahqg9o6ky82k1ryxbcrmlfz8vg0 2448160 2448159 2026-08-17T12:30:29Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1170: Depool db1170.eqiad.wmnet to then clone it to db1284.eqiad.wmnet - marostegui@cumin1003 2448160 wikitext text/x-wiki ezgxt0fr0637khl9gzmnjpvi2b8ebrl 2448162 2448160 2026-08-17T12:31:57Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-maint1001.eqiad.wmnet 2448162 wikitext text/x-wiki p4s78pvelj8ryumtf2cu4sjf2qm0n5n 2448163 2448162 2026-08-17T12:32:14Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1028.eqiad.wmnet 2448163 wikitext text/x-wiki 6rvyb97zmsfddqt2qc1om015sej3j3a 2448164 2448163 2026-08-17T12:32:19Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1028.eqiad.wmnet 2448164 wikitext text/x-wiki ojx3qpkpfi0jkemozutixf5orip0r6r 2448166 2448164 2026-08-17T12:34:38Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2190-2203].codfw.wmnet 2448166 wikitext text/x-wiki m23l0uvkvilo0ivx0ll82v662sqoqbw 2448167 2448166 2026-08-17T12:34:41Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2190-2203].codfw.wmnet 2448167 wikitext text/x-wiki kxzmkj32cvtubvkhmmwo9l4ta4cezgd 2448168 2448167 2026-08-17T12:35:08Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2204-2215,2242-2243].codfw.wmnet 2448168 wikitext text/x-wiki f5ookkf79vhv9isn4n9bk09lt598hs5 2448169 2448168 2026-08-17T12:35:47Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1324832|Migrate $wgFlaggedRevsTags from flaggedrevs.php to ext-FlaggedRevs.php]] 2448169 wikitext text/x-wiki b789lewbi9ksq2fuj2wb3rp1fb6d3os 2448171 2448169 2026-08-17T12:37:53Z Stashbot 7414 seanleong-wmde: Finished populateSitesTable for [bolwiki] ([T429955]) 2448171 wikitext text/x-wiki jcptgiufgkm1c2fk2ia6zf7vn5kawja 2448172 2448171 2026-08-17T12:37:57Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1324832|Migrate $wgFlaggedRevsTags from flaggedrevs.php to ext-FlaggedRevs.php]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448172 wikitext text/x-wiki 8oxwv5cy8vpw0dpj2qz5veo7z1xqvxx 2448173 2448172 2026-08-17T12:40:03Z Stashbot 7414 ladsgroup@deploy1003: Rolling back deployment 2448173 wikitext text/x-wiki g5sopwxepvrx8vbhv12bj567nwbg9dm 2448174 2448173 2026-08-17T12:41:22Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1037.eqiad.wmnet 2448174 wikitext text/x-wiki 5xqaayyi89f2bsxp8uglhi48jo2yxgt 2448175 2448174 2026-08-17T12:41:37Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ldap-maint2001.codfw.wmnet 2448175 wikitext text/x-wiki 8w9zxo3hjz6sq385dra2o3qd7cy1byw 2448176 2448175 2026-08-17T12:43:14Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2204-2215,2242-2243].codfw.wmnet 2448176 wikitext text/x-wiki bsms6p7jufy0ka48dttgk06lx93hog8 2448177 2448176 2026-08-17T12:45:27Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-maint2001.codfw.wmnet 2448177 wikitext text/x-wiki puv5b83qxz48dyn17qqtklul0nro83j 2448179 2448177 2026-08-17T12:49:50Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324832|Migrate $wgFlaggedRevsTags from flaggedrevs.php to ext-FlaggedRevs.php]] (duration: 14m 02s) 2448179 wikitext text/x-wiki o5ichntwo1ugg9kaki64n63m4mydu8k 2448180 2448179 2026-08-17T12:51:21Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2204-2215,2242-2243].codfw.wmnet 2448180 wikitext text/x-wiki asfivwikwr9i53hcb516soej5qsy2lz 2448181 2448180 2026-08-17T12:51:24Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2204-2215,2242-2243].codfw.wmnet 2448181 wikitext text/x-wiki om9w4teqlklryh6xdnk4xerabq4vcih 2448182 2448181 2026-08-17T12:51:54Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2248-2261].codfw.wmnet 2448182 wikitext text/x-wiki kxqfb9jwpt6nqj9zwoy7pkglngvkpm7 2448184 2448182 2026-08-17T13:00:24Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1037.eqiad.wmnet 2448184 wikitext text/x-wiki 7h9cu6g9gmgt4ysmbdjis5kmvbmhoma 2448186 2448184 2026-08-17T13:04:41Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2248-2261].codfw.wmnet 2448186 wikitext text/x-wiki lf9zrv4075xikcgy8m2l1ttk1r37lit 2448187 2448186 2026-08-17T13:06:27Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2448187 wikitext text/x-wiki 1qqf3nve3jkx4pq6g8fj6ojdhxxjsj1 2448188 2448187 2026-08-17T13:08:21Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Started scap sync-world: Backport for [[gerrit:1311141|viwiki: Set `noindex,nofollow` for User and User talk (T432311)]] 2448188 wikitext text/x-wiki 106u60egul2b3dd7l0y0kwzm1gk1f6n 2448189 2448188 2026-08-17T13:08:25Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1037.eqiad.wmnet 2448189 wikitext text/x-wiki fwfpj2vciiq77bb7sc74ijt1sr1aihs 2448190 2448189 2026-08-17T13:08:27Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1037.eqiad.wmnet 2448190 wikitext text/x-wiki snv0qewdpl29x1n2bbn6yx0lsc9d2pr 2448191 2448190 2026-08-17T13:10:12Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: ndkdd, lucaswerkmeister-wmde: Backport for [[gerrit:1311141|viwiki: Set `noindex,nofollow` for User and User talk (T432311)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448191 wikitext text/x-wiki c1xda6pilzfkl2b4x0ypdjh6wit8x70 2448192 2448191 2026-08-17T13:10:45Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1049] - vriley@cumin1003" 2448192 wikitext text/x-wiki nagjen1qe9ltxn5g1fcrq1i2fouq4m7 2448193 2448192 2026-08-17T13:10:51Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1049] - vriley@cumin1003" 2448193 wikitext text/x-wiki rwh9em9pvy4ywv3qawyrrxq0ldiaus3 2448194 2448193 2026-08-17T13:10:54Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448194 wikitext text/x-wiki ghu4xqkp2nxpe0q4h1xvh5qattabf8c 2448195 2448194 2026-08-17T13:11:48Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1049 2448195 wikitext text/x-wiki rdv1p8evxr4n2m5h39pk7xp81bqf4a7 2448196 2448195 2026-08-17T13:12:08Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1049 2448196 wikitext text/x-wiki 1og9b5fz5pvnzdc8kba8cwtderk5wlu 2448197 2448196 2026-08-17T13:13:09Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1049.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448197 wikitext text/x-wiki f0h2oc4ytlg27yvdia66whppxvaiqwh 2448198 2448197 2026-08-17T13:14:35Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2248-2261].codfw.wmnet 2448198 wikitext text/x-wiki mji4xcaeuahb5meayxxe36i4tw94079 2448199 2448198 2026-08-17T13:14:38Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2248-2261].codfw.wmnet 2448199 wikitext text/x-wiki b9pg8grc40wwm638r27e5riwkyjktjr 2448200 2448199 2026-08-17T13:14:48Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: ndkdd, lucaswerkmeister-wmde: Continuing with deployment 2448200 wikitext text/x-wiki ij2yzm8208u4hvutvfca6x0z2orl9ov 2448201 2448200 2026-08-17T13:15:04Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2262-2275].codfw.wmnet 2448201 wikitext text/x-wiki 2ceuayxm0i9x6iwu5kczivebwxqfngk 2448202 2448201 2026-08-17T13:19:32Z Stashbot 7414 lucaswerkmeister-wmde@deploy1003: Finished scap sync-world: Backport for [[gerrit:1311141|viwiki: Set `noindex,nofollow` for User and User talk (T432311)]] (duration: 11m 11s) 2448202 wikitext text/x-wiki my1ovjosuq00p23y3xaxlna7dn8rxvb 2448203 2448202 2026-08-17T13:21:43Z Stashbot 7414 atsukoito: manually done sudo -i docker-registryctl --debug delete-tags 'docker-registry.discovery.wmnet/repos/data-engineering/airflow-dags:airflow-3.3.0-py3.11-2026-08-17-*' to remove incorrect tags 2448203 wikitext text/x-wiki g94nkvbz5lub7lasucoaciknhqa8z19 2448204 2448203 2026-08-17T13:23:27Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1045.eqiad.wmnet 2448204 wikitext text/x-wiki fwik3bmrez9k1gd5jwz2jc25s95y67u 2448205 2448204 2026-08-17T13:23:57Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1049.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448205 wikitext text/x-wiki q7n3gter675k8ukxp4bz41b88ae8hku 2448206 2448205 2026-08-17T13:26:10Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1170: Pool db1170.eqiad.wmnet in after cloning 2448206 wikitext text/x-wiki qamfhdbk7ganil8rkicsar7yyoah6k6 2448207 2448206 2026-08-17T13:26:12Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1049.eqiad.wmnet with OS trixie 2448207 wikitext text/x-wiki 2pflvzfk6v5wabqda1669i95vu6msby 2448208 2448207 2026-08-17T13:26:32Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1049.eqiad.wmnet with OS trixie 2448208 wikitext text/x-wiki 7xw0tm8s2qom6vtugg5sxy21j6tuqch 2448209 2448208 2026-08-17T13:27:14Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2262-2275].codfw.wmnet 2448209 wikitext text/x-wiki cw476qor6mzyrb1w1i9wyhjjb4i0rm1 2448210 2448209 2026-08-17T13:30:49Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1045.eqiad.wmnet 2448210 wikitext text/x-wiki 5udqizbrvulkr7du0s1btxnmd5211i5 2448212 2448210 2026-08-17T13:32:35Z Stashbot 7414 Lucas_WMDE: UTC afternoon backport+config window done 2448212 wikitext text/x-wiki rt3svf97lle5qh9c3wjtihes6rww7sh 2448213 2448212 2026-08-17T13:34:30Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2262-2275].codfw.wmnet 2448213 wikitext text/x-wiki 0sla579vhrf498utwuq6vv71ce9unrf 2448214 2448213 2026-08-17T13:34:33Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2262-2275].codfw.wmnet 2448214 wikitext text/x-wiki 8650d1ii3ql1yc8s5w8fh4lme5pmo6d 2448215 2448214 2026-08-17T13:34:47Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1049.eqiad.wmnet with OS trixie 2448215 wikitext text/x-wiki 6woro41v5zburnzzwqn7xvbse933kdv 2448216 2448215 2026-08-17T13:35:00Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2276-2289].codfw.wmnet 2448216 wikitext text/x-wiki 3g7ykqfb37hcokw68t2ofqe8xd3j6a5 2448217 2448216 2026-08-17T13:36:08Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1045.eqiad.wmnet 2448217 wikitext text/x-wiki 76sn09mnlh6cu6goqs1h99xnmxznwcg 2448218 2448217 2026-08-17T13:36:14Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1045.eqiad.wmnet 2448218 wikitext text/x-wiki h5z1kn3sank1m06yqlstsuphr1u1of9 2448219 2448218 2026-08-17T13:36:36Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1046.eqiad.wmnet 2448219 wikitext text/x-wiki mrzq8r6nyrl6x8fjoj7a7829d70kswo 2448221 2448219 2026-08-17T13:43:17Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2276-2289].codfw.wmnet 2448221 wikitext text/x-wiki 0h7t9bv5t1ghjprtqe6ch2cmnz59abd 2448222 2448221 2026-08-17T13:43:59Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1046.eqiad.wmnet 2448222 wikitext text/x-wiki 8waeztoukcpdenw2nzgquqt4h6vrxxl 2448223 2448222 2026-08-17T13:44:26Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-analytics-external: apply 2448223 wikitext text/x-wiki geoasufqpzrlm7z5l50bpycl0sy0jqx 2448224 2448223 2026-08-17T13:46:18Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-analytics-external: apply 2448224 wikitext text/x-wiki g5qq92ux0klf3l2n084jo3ar5nvia8i 2448225 2448224 2026-08-17T13:48:02Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.cdn.roll-restart-reboot-hcaptcha-proxy rolling reboot on A:hcaptcha-proxy and A:hcaptcha-proxy 2448225 wikitext text/x-wiki 8195f1vzzsrsx0r5h9w4s7qk4owplib 2448227 2448225 2026-08-17T13:48:43Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.cdn.roll-restart-reboot-ncredir rolling reboot on A:ncredir and A:ncredir 2448227 wikitext text/x-wiki j41xpllff3bbq7dv5u20iv6pz72hq49 2448228 2448227 2026-08-17T13:49:22Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1046.eqiad.wmnet 2448228 wikitext text/x-wiki c3ey3yju365hmakt8jd7xwntykl31o9 2448229 2448228 2026-08-17T13:49:41Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1049.eqiad.wmnet with reason: host reimage 2448229 wikitext text/x-wiki onckqorxo4863foq9sf3lhre7eyjmi8 2448230 2448229 2026-08-17T13:49:48Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1046.eqiad.wmnet 2448230 wikitext text/x-wiki fr38k0dtg3zy7wg3nlz611107k0ggvc 2448231 2448230 2026-08-17T13:49:57Z Stashbot 7414 phuedx@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-analytics-external: apply 2448231 wikitext text/x-wiki qnyjfypq2c7qdsmdvcvprvyb3wi6zaq 2448232 2448231 2026-08-17T13:50:30Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2276-2289].codfw.wmnet 2448232 wikitext text/x-wiki rz0efawfzwzn48gtxmja9dd2g5kawkd 2448233 2448232 2026-08-17T13:50:33Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2276-2289].codfw.wmnet 2448233 wikitext text/x-wiki 4xb41noh4ik83qxjbamqoetu13zqimr 2448234 2448233 2026-08-17T13:50:54Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1047.eqiad.wmnet 2448234 wikitext text/x-wiki 54lququdzjb3ua0kxpels9vl1ccfyd9 2448235 2448234 2026-08-17T13:51:02Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2290-2303].codfw.wmnet 2448235 wikitext text/x-wiki taf6thz1yk19o5q27oxux9db6u4vhy8 2448236 2448235 2026-08-17T13:51:09Z Stashbot 7414 phuedx@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-analytics-external: apply 2448236 wikitext text/x-wiki 7yxpl9v4kuztxxlrkem9dorf8lfjt1y 2448237 2448236 2026-08-17T13:52:51Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1049.eqiad.wmnet with reason: host reimage 2448237 wikitext text/x-wiki 92xr35co2pb2g8u618ygkpfr7bxv0sj 2448238 2448237 2026-08-17T13:53:38Z Stashbot 7414 phuedx@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-analytics-external: apply 2448238 wikitext text/x-wiki rfngpq5198on0dt624ssqucpge941hu 2448239 2448238 2026-08-17T13:54:22Z Stashbot 7414 phuedx@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics-external: apply 2448239 wikitext text/x-wiki klfa1p70x6miun0ent83b63l7myrcj8 2448240 2448239 2026-08-17T13:54:33Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2448240 wikitext text/x-wiki tpd33091w98xx5hwoo7it4tjczdpjzq 2448241 2448240 2026-08-17T13:54:55Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2448241 wikitext text/x-wiki 6ejqh2wcyk7n1ntl61o985l03ji01dw 2448242 2448241 2026-08-17T13:55:01Z Stashbot 7414 dbrant@deploy1003: helmfile [staging] START helmfile.d/services/wikifeeds: apply 2448242 wikitext text/x-wiki sv83zzyecksbl3o0hrnhg3ckgp4g68o 2448243 2448242 2026-08-17T13:56:00Z Stashbot 7414 dbrant@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifeeds: apply 2448243 wikitext text/x-wiki hv03gzdfg9c3wmiacpzp6y3lqtpn89b 2448244 2448243 2026-08-17T13:56:27Z Stashbot 7414 dbrant@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifeeds: apply 2448244 wikitext text/x-wiki r9bgic7ybt14g1njwnic8m76x3qzi93 2448245 2448244 2026-08-17T13:56:37Z Stashbot 7414 cjd91: sudo -i cookbook sre.cdn.roll-upgrade-ats --query 'A:cp-ulsfo or A:cp-drmrs' --task-id T434620 --reason '9.2.15 upgrade' 2448245 wikitext text/x-wiki drjo82mgqbyypfpoaer764amtaeg24a 2448246 2448245 2026-08-17T13:56:42Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-upgrade-ats Rolling upgrade of ATS on A:cp-ulsfo or A:cp-drmrs and A:cp - 9.2.15 upgrade (T434620) 2448246 wikitext text/x-wiki kwa8xm906yo4vdd7woy8dqwwj4iossa 2448247 2448246 2026-08-17T13:56:56Z Stashbot 7414 dbrant@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifeeds: apply 2448247 wikitext text/x-wiki m6r23vzw4jzic08732zc4qj9lor4e3h 2448248 2448247 2026-08-17T13:57:07Z Stashbot 7414 dbrant@deploy1003: helmfile [codfw] START helmfile.d/services/wikifeeds: apply 2448248 wikitext text/x-wiki 0ik8p5jykqpxmcs50apc9jbuy81tujz 2448249 2448248 2026-08-17T13:57:38Z Stashbot 7414 dbrant@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply 2448249 wikitext text/x-wiki h2ubijs2vg0m7z1edieukdp7bfy1o76 2448250 2448249 2026-08-17T13:59:08Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2448250 wikitext text/x-wiki rz728j5kd2vjvd0z1vah3acpxaifnh5 2448251 2448250 2026-08-17T14:00:02Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1143.eqiad.wmnet with OS bookworm 2448251 wikitext text/x-wiki 5m2c7d24krofeot5q7yzjjv9wqu8svn 2448252 2448251 2026-08-17T14:00:05Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1182.eqiad.wmnet with OS bookworm 2448252 wikitext text/x-wiki 7ekmbe0429v8mxtuxdg53gsesfl7xpd 2448253 2448252 2026-08-17T14:01:50Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2448253 wikitext text/x-wiki s5p1571x1fpfzfiei0e0uw7d8bcyoge 2448254 2448253 2026-08-17T14:02:55Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1188.eqiad.wmnet with OS bookworm 2448254 wikitext text/x-wiki t8by7g8dxeh0kw93d5stk4j2leiyfz2 2448255 2448254 2026-08-17T14:02:59Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1189.eqiad.wmnet with OS bookworm 2448255 wikitext text/x-wiki 4rydw35m9l79e344bozmbtu3s32gmuq 2448256 2448255 2026-08-17T14:04:17Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2290-2303].codfw.wmnet 2448256 wikitext text/x-wiki f9efurj0kk9lubet5pue4mgj4t8hnjw 2448257 2448256 2026-08-17T14:04:26Z Stashbot 7414 klausman@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2448257 wikitext text/x-wiki tbdmn023yjqtsancpfr60b1407ziodt 2448258 2448257 2026-08-17T14:04:45Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2448258 wikitext text/x-wiki 3bqutk3f0cpaop9idbq9y6vo2gr3bki 2448259 2448258 2026-08-17T14:06:42Z Stashbot 7414 klausman@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2448259 wikitext text/x-wiki 5bpu3slv41toihjy4bp0ew6523jzndt 2448261 2448259 2026-08-17T14:09:01Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1051] - vriley@cumin1003" 2448261 wikitext text/x-wiki cws56pjnue8fb293w14jpd75y5bn9dw 2448262 2448261 2026-08-17T14:09:06Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1051] - vriley@cumin1003" 2448262 wikitext text/x-wiki 8r5phjl6iqor297n1m3dt9d3e1d4fpk 2448263 2448262 2026-08-17T14:09:09Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448263 wikitext text/x-wiki nkbqw4kvrr17r25efefkc8zl263nv4v 2448264 2448263 2026-08-17T14:09:56Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1003" 2448264 wikitext text/x-wiki guuulcb468extw8rupp6b0lf6sk1jt2 2448265 2448264 2026-08-17T14:11:23Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1170: Pool db1170.eqiad.wmnet in after cloning 2448265 wikitext text/x-wiki e2myvofbhnacssaz3ubzhlxdve1igtk 2448266 2448265 2026-08-17T14:11:32Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1170.eqiad.wmnet onto db1284.eqiad.wmnet 2448266 wikitext text/x-wiki 4jsz8ve6x1pnxo5btmc1rjmyt7wcoox 2448267 2448266 2026-08-17T14:11:38Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1051 2448267 wikitext text/x-wiki l03vxoq7isu1b00b40knxklp3cy3qy2 2448268 2448267 2026-08-17T14:11:54Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1051 2448268 wikitext text/x-wiki cnx1zrava2xybjq0o8b342m51pf6tp4 2448269 2448268 2026-08-17T14:12:07Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1003" 2448269 wikitext text/x-wiki bro3wn8gmegsju1no2fjnkuzb550t92 2448270 2448269 2026-08-17T14:12:10Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2290-2303].codfw.wmnet 2448270 wikitext text/x-wiki 0cbvuxzsc8zyfsbq9mlhrtgtatncxd4 2448271 2448270 2026-08-17T14:12:13Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1049.eqiad.wmnet with OS trixie 2448271 wikitext text/x-wiki ez9bzzr5py6ih66q680lqva7b4y65tm 2448272 2448271 2026-08-17T14:12:16Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2290-2303].codfw.wmnet 2448272 wikitext text/x-wiki ta9usx57eny043czen5n6txfq12esnd 2448273 2448272 2026-08-17T14:12:33Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2304-2317].codfw.wmnet 2448273 wikitext text/x-wiki 9o0uozbuaklnfb1cg8rn74rnht4rqr3 2448274 2448273 2026-08-17T14:13:07Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1051.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448274 wikitext text/x-wiki hzzjrrqtzubuiyptco74525pwkepati 2448275 2448274 2026-08-17T14:14:56Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1143.eqiad.wmnet with reason: host reimage 2448275 wikitext text/x-wiki 74s8acsy2drz4oksr8k54pt9tq7tnat 2448276 2448275 2026-08-17T14:15:16Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1182.eqiad.wmnet with reason: host reimage 2448276 wikitext text/x-wiki 26ibnnxauy2ofanh6q962wa1qmzmi9d 2448277 2448276 2026-08-17T14:17:30Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1189.eqiad.wmnet with reason: host reimage 2448277 wikitext text/x-wiki jypapjvp2x0y1l8bsbbyi46hj15f0od 2448278 2448277 2026-08-17T14:19:04Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host dns1006.wikimedia.org with OS trixie 2448278 wikitext text/x-wiki 59uxngyox1qj0elvrt00lyy5guhq0em 2448279 2448278 2026-08-17T14:19:07Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1143.eqiad.wmnet with reason: host reimage 2448279 wikitext text/x-wiki qlocshzbm37k5nu1d9fxucu9mi2u17k 2448280 2448279 2026-08-17T14:19:15Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host dns2006.wikimedia.org with OS trixie 2448280 wikitext text/x-wiki akiqo6bvaevi8o7bbxanldb95bd02cq 2448281 2448280 2026-08-17T14:19:30Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host dns4004.wikimedia.org with OS trixie 2448281 wikitext text/x-wiki mk81honshkluky4fjrarbn5i0cseyvk 2448282 2448281 2026-08-17T14:20:16Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2304-2317].codfw.wmnet 2448282 wikitext text/x-wiki fx2zwgjunht1swh3k4qmvyg506ea0p9 2448283 2448282 2026-08-17T14:21:20Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2448283 wikitext text/x-wiki n571eytgjwxwy4odjefw25pgrw9wmfc 2448284 2448283 2026-08-17T14:21:53Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reboot-single for host acmechief-test1001.eqiad.wmnet 2448284 wikitext text/x-wiki 2ynn090pc1tpb2wa19120udp246fbvz 2448285 2448284 2026-08-17T14:22:11Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reboot-single for host acmechief-test2001.codfw.wmnet 2448285 wikitext text/x-wiki g43ylue4rmh8ftdc7qt6ejx1qagizbt 2448286 2448285 2026-08-17T14:22:15Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2448286 wikitext text/x-wiki 34szvn5kpe6x4f55vch4k25gjz5cj3z 2448287 2448286 2026-08-17T14:22:21Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1051.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448287 wikitext text/x-wiki gam3hotjzg89tu4cf0noqyfesoyvtgr 2448288 2448287 2026-08-17T14:22:41Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1182.eqiad.wmnet with reason: host reimage 2448288 wikitext text/x-wiki p47vjjzjppks4i388bg48thdvtofri1 2448289 2448288 2026-08-17T14:23:35Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2448289 wikitext text/x-wiki 385wtbjtzmbpry8ewsegfbr75jeho5z 2448290 2448289 2026-08-17T14:24:21Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1051.eqiad.wmnet with OS trixie 2448290 wikitext text/x-wiki aw1o6tefwrtfme3yeogcmreakaey0sl 2448292 2448290 2026-08-17T14:25:33Z Stashbot 7414 claime: Deploying wmgRedisLockPassword - T366938 T427999 2448292 wikitext text/x-wiki lengyqzgzlanmqqlrlw9vzz78ik9j34 2448293 2448292 2026-08-17T14:25:37Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host acmechief-test1001.eqiad.wmnet 2448293 wikitext text/x-wiki f169ikbyz9qe6kazgv84rxfzya833r8 2448294 2448293 2026-08-17T14:26:09Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host acmechief-test2001.codfw.wmnet 2448294 wikitext text/x-wiki cig5l8xbvnmao3f1gd56b96tq1e2uqs 2448295 2448294 2026-08-17T14:26:47Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1189.eqiad.wmnet with reason: host reimage 2448295 wikitext text/x-wiki bb5lpwgbmio1blnps07r40d0v74mqb2 2448296 2448295 2026-08-17T14:27:20Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1054] - vriley@cumin1003" 2448296 wikitext text/x-wiki sximsuyagrbp7d5jgzme1ox3upnh9bw 2448297 2448296 2026-08-17T14:27:25Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1054] - vriley@cumin1003" 2448297 wikitext text/x-wiki mmysb7vp0twyu4cist58b24odgfxqvs 2448298 2448297 2026-08-17T14:27:28Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448298 wikitext text/x-wiki k0a0pt56rddsc0w58fwrlm0bs06ot95 2448299 2448298 2026-08-17T14:27:43Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1054 2448299 wikitext text/x-wiki sy1g5jj9hbq3h08qvz1l59dn7zbb1xg 2448300 2448299 2026-08-17T14:28:02Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1054 2448300 wikitext text/x-wiki 7p32xocstco3vbbcmyptzbxu3e3yfxx 2448301 2448300 2026-08-17T14:28:06Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2304-2317].codfw.wmnet 2448301 wikitext text/x-wiki swoytvfu6xpikvncc39ndowtdsm56ah 2448302 2448301 2026-08-17T14:28:09Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2304-2317].codfw.wmnet 2448302 wikitext text/x-wiki 7czos79ihqi389eg4djwj662tpbf40t 2448303 2448302 2026-08-17T14:28:38Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2318-2331].codfw.wmnet 2448303 wikitext text/x-wiki 7ut1ann2komo9xiyqb35m72tb7cbgwz 2448304 2448303 2026-08-17T14:29:20Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1054.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448304 wikitext text/x-wiki djzmxacf3462y4z9h4taktgzqeak3rl 2448306 2448304 2026-08-17T14:34:43Z Stashbot 7414 cgoubert@deploy1003: Started scap sync-world: Backport for [[gerrit:1326306|Add placeholder $wmgRedisLockPassword (T366938 T427999)]] 2448306 wikitext text/x-wiki m41tmj93vgsq11nwi7zn84udrmejgra 2448307 2448306 2026-08-17T14:36:25Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2318-2331].codfw.wmnet 2448307 wikitext text/x-wiki h55zpmer3gmkqleq8s8diguauabdp3w 2448308 2448307 2026-08-17T14:36:29Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reboot-single for host acmechief1002.eqiad.wmnet 2448308 wikitext text/x-wiki q0dufg2kzejt539cyopq5trlxt2n38c 2448309 2448308 2026-08-17T14:36:34Z Stashbot 7414 cgoubert@deploy1003: cgoubert: Backport for [[gerrit:1326306|Add placeholder $wmgRedisLockPassword (T366938 T427999)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448309 wikitext text/x-wiki i3ib2mgtutpgpf4vjbbcwy296p989pl 2448310 2448309 2026-08-17T14:37:20Z Stashbot 7414 cgoubert@deploy1003: cgoubert: Continuing with deployment 2448310 wikitext text/x-wiki 4h8z6tvs8l4dne4f617k8ox8iefd7d8 2448311 2448310 2026-08-17T14:37:30Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on dns1006.wikimedia.org with reason: host reimage 2448311 wikitext text/x-wiki mm2gowrvm3xwyqbh32u3m895105re0h 2448314 2448311 2026-08-17T14:38:37Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on dns2006.wikimedia.org with reason: host reimage 2448314 wikitext text/x-wiki iok6d0022m4zgkyhlfhii23nsse4u1f 2448316 2448314 2026-08-17T14:39:24Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1051.eqiad.wmnet with reason: host reimage 2448316 wikitext text/x-wiki t4fxr8gs1yd673kje2lykeindcf8bok 2448317 2448316 2026-08-17T14:40:12Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host acmechief1002.eqiad.wmnet 2448317 wikitext text/x-wiki 2rnvlvd3zr9v6fsxxqgsbcl3jl0psqt 2448318 2448317 2026-08-17T14:41:29Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reboot-single for host acmechief2002.codfw.wmnet 2448318 wikitext text/x-wiki 3dfcaz3elv4f3sot4f9b1wyaftmzazh 2448319 2448318 2026-08-17T14:41:39Z Stashbot 7414 cgoubert@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326306|Add placeholder $wmgRedisLockPassword (T366938 T427999)]] (duration: 06m 56s) 2448319 wikitext text/x-wiki s994aezywjf6y6s3g5y0aar89rzoamm 2448320 2448319 2026-08-17T14:42:48Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.cassandra.roll-restart for nodes matching aqs[2002-2012].codfw.wmnet,aqs[1017-1027].eqiad.wmnet: Upgrade Cassandra to 5.0.8 — T433026 - eevans@cumin1003 2448320 wikitext text/x-wiki cq6c9puthtyvkr7cxjfc3kgghgh2e0f 2448321 2448320 2026-08-17T14:43:40Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2318-2331].codfw.wmnet 2448321 wikitext text/x-wiki me2hco4mqk9rk868h6apjskjao6xiny 2448322 2448321 2026-08-17T14:43:43Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2318-2331].codfw.wmnet 2448322 wikitext text/x-wiki r2u5dkaqt1n0xrdxlvtbt6mkpm80pl7 2448323 2448322 2026-08-17T14:43:47Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2016.codfw.wmnet 2448323 wikitext text/x-wiki gchnuztsyo6pffne8xtxmdltxwtxc0e 2448324 2448323 2026-08-17T14:43:56Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1143.eqiad.wmnet with OS bookworm 2448324 wikitext text/x-wiki bwxadudgnm83sgrtm3r0vwx8podelmq 2448325 2448324 2026-08-17T14:44:22Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker2016.codfw.wmnet 2448325 wikitext text/x-wiki qe41zrm244mds92hw8a1rqpuehsmxtc 2448326 2448325 2026-08-17T14:44:54Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns1006.wikimedia.org with reason: host reimage 2448326 wikitext text/x-wiki ikc4yffi7yub8j3t1vntmsdehcbc8hl 2448327 2448326 2026-08-17T14:45:18Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host acmechief2002.codfw.wmnet 2448327 wikitext text/x-wiki e2qaljjxv11etytcemidvv06dneblu7 2448328 2448327 2026-08-17T14:46:56Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1182.eqiad.wmnet with OS bookworm 2448328 wikitext text/x-wiki ebib9sc0hm8qy5u6wq25fm44abfjwrc 2448329 2448328 2026-08-17T14:48:19Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1051.eqiad.wmnet with reason: host reimage 2448329 wikitext text/x-wiki 4g1x6fyybx724swlx29y9xfcm69oq2e 2448330 2448329 2026-08-17T14:48:32Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1188.eqiad.wmnet with reason: host reimage 2448330 wikitext text/x-wiki 1icef68vn8i86hpodrfwgptxpesoqej 2448331 2448330 2026-08-17T14:50:19Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1189.eqiad.wmnet with OS bookworm 2448331 wikitext text/x-wiki 0f07lfvnply6h850eq3ew9n6qswria0 2448332 2448331 2026-08-17T14:50:57Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2016.codfw.wmnet 2448332 wikitext text/x-wiki 3vk4kx7akcvinph15gin1up0exszwd0 2448333 2448332 2026-08-17T14:51:00Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2016.codfw.wmnet 2448333 wikitext text/x-wiki c75hq1e0wb24kntlifmeez5dfg4nosz 2448334 2448333 2026-08-17T14:51:03Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker-exp2001.codfw.wmnet 2448334 wikitext text/x-wiki j3u2zjc0ixqrwfw8a39c0u4r8zxhfg1 2448335 2448334 2026-08-17T14:51:06Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2448335 wikitext text/x-wiki k3rpyqn2zb77s9kkfaz28zupnfubjp0 2448336 2448335 2026-08-17T14:51:33Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker-exp2001.codfw.wmnet 2448336 wikitext text/x-wiki ilicloxbvk6gae546tvyr8ng7xg0cdr 2448337 2448336 2026-08-17T14:52:25Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1188.eqiad.wmnet with reason: host reimage 2448337 wikitext text/x-wiki f0lc7d97n7iyq6hl6evgmsjin5n4i94 2448338 2448337 2026-08-17T14:54:04Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1047.eqiad.wmnet 2448338 wikitext text/x-wiki ow0b005i1b7gj1s0dum6r5gwvkf7rge 2448339 2448338 2026-08-17T14:55:42Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1055] - vriley@cumin1003" 2448339 wikitext text/x-wiki 4433begv5zxu3tsruy6qkgbg1plv64y 2448340 2448339 2026-08-17T14:55:47Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1055] - vriley@cumin1003" 2448340 wikitext text/x-wiki 6zs5wxnvbhcffox6qyw7l5cs36z6kj4 2448341 2448340 2026-08-17T14:55:50Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448341 wikitext text/x-wiki lu3gh17w2nb8g4h50k0dfwjymdsd518 2448342 2448341 2026-08-17T14:55:53Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns2006.wikimedia.org with reason: host reimage 2448342 wikitext text/x-wiki k0bupr35f9ws9vglizykljshp47mzv6 2448343 2448342 2026-08-17T14:56:18Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1055 2448343 wikitext text/x-wiki 6aeq5t2n6bt1idopybqcyul3t0nk97x 2448344 2448343 2026-08-17T14:56:33Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1055 2448344 wikitext text/x-wiki afmtdhied84nw6lunq66sco4dm0pnem 2448345 2448344 2026-08-17T14:57:23Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448345 wikitext text/x-wiki slib13p5ran1x8tc9fbrd5s092ripxk 2448346 2448345 2026-08-17T14:57:25Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.admin rebooting A:liberica-magru and not P{lvs7003*} and A:liberica 2448346 wikitext text/x-wiki is2sdsul5j85q21m0jt4kqyyziecbir 2448347 2448346 2026-08-17T14:57:41Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker-exp2001.codfw.wmnet 2448347 wikitext text/x-wiki ha89sbhuvtu32btm9zn98rml707bvss 2448348 2448347 2026-08-17T14:57:43Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker-exp2001.codfw.wmnet 2448348 wikitext text/x-wiki mcdk5cscqr0mj4k9nb3swuowt3dno6x 2448349 2448348 2026-08-17T14:57:45Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on A:wikikube-worker-codfw 2448349 wikitext text/x-wiki g98g1svkyo85oz1ymkkpu432v8ep83i 2448350 2448349 2026-08-17T14:58:47Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-reboot-hcaptcha-proxy (exit_code=0) rolling reboot on A:hcaptcha-proxy and A:hcaptcha-proxy 2448350 wikitext text/x-wiki sjxn14bxycddyfaj58w0bcvhptd06tu 2448351 2448350 2026-08-17T14:59:35Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1047.eqiad.wmnet 2448351 wikitext text/x-wiki fbd0lcmnpmi7xnhuj03oofnuwvno7tg 2448352 2448351 2026-08-17T15:00:00Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1047.eqiad.wmnet 2448352 wikitext text/x-wiki rl5flqosjvx4m5m9godv0mmqrcb9dg1 2448353 2448352 2026-08-17T15:00:20Z Stashbot 7414 taavi@deploy1003: Started scap sync-world: Backport for [[gerrit:1318203|Undeploy WP25EasterEggs (II) (T418134)]] 2448353 wikitext text/x-wiki bgo5vckq722loz028nodjodh9ecf0p5 2448354 2448353 2026-08-17T15:02:25Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448354 wikitext text/x-wiki ofibwsmvvcbxq3gwk9y9qphwv2t0usi 2448355 2448354 2026-08-17T15:03:29Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1055.eqiad.wmnet with OS trixie 2448355 wikitext text/x-wiki 398tto6dkvvqvoe7bxt2g8vzgya4cd8 2448356 2448355 2026-08-17T15:04:19Z Stashbot 7414 taavi@deploy1003: taavi: Backport for [[gerrit:1318203|Undeploy WP25EasterEggs (II) (T418134)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448356 wikitext text/x-wiki 2s8njzq9bkhyd2tmbc33kcklisebqjl 2448357 2448356 2026-08-17T15:04:25Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.cdn.roll-restart-reboot-ncredir (exit_code=0) rolling reboot on A:ncredir and A:ncredir 2448357 wikitext text/x-wiki aut2dfktr4beraer9sis6k152kaqqs5 2448358 2448357 2026-08-17T15:05:02Z Stashbot 7414 taavi@deploy1003: taavi: Continuing with deployment 2448358 wikitext text/x-wiki 69urn3wf4g49jjt3vtp8mi8y1r0ecxk 2448359 2448358 2026-08-17T15:05:25Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1003" 2448359 wikitext text/x-wiki lbgx60lk392bxtmmle71k985ur461kb 2448360 2448359 2026-08-17T15:06:15Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - vriley@cumin1003" 2448360 wikitext text/x-wiki fgmjywmqipsme0zzhk0fxc67bgflrtb 2448361 2448360 2026-08-17T15:06:18Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1051.eqiad.wmnet with OS trixie 2448361 wikitext text/x-wiki pa32yqt58npsj0w4e1o07wqcvehj5sy 2448362 2448361 2026-08-17T15:06:56Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) rebooting A:liberica-magru and not P{lvs7003*} and A:liberica 2448362 wikitext text/x-wiki prlq6dqx3uotp2ovfld5364882p6z35 2448363 2448362 2026-08-17T15:09:17Z Stashbot 7414 taavi@deploy1003: Finished scap sync-world: Backport for [[gerrit:1318203|Undeploy WP25EasterEggs (II) (T418134)]] (duration: 08m 56s) 2448363 wikitext text/x-wiki 4r2lysnch6yquh81nevd2qe6slyapr2 2448364 2448363 2026-08-17T15:12:54Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1188.eqiad.wmnet with OS bookworm 2448364 wikitext text/x-wiki bt7enaludv9zmfaz0f0gnw6mo8xm6ce 2448365 2448364 2026-08-17T15:13:24Z Stashbot 7414 sukhe@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host dns4004.wikimedia.org with OS trixie 2448365 wikitext text/x-wiki 7csdmznkxn000q77bjfsiznf05gwko8 2448366 2448365 2026-08-17T15:13:49Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2448366 wikitext text/x-wiki 0thx3b30g903a859cmuqcydic35pbbj 2448367 2448366 2026-08-17T15:16:22Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.admin rebooting A:liberica-ulsfo and A:liberica 2448367 wikitext text/x-wiki g34ba7p6g2dfduerj3yd617f2uonkfj 2448368 2448367 2026-08-17T15:18:09Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1154.eqiad.wmnet with OS bookworm 2448368 wikitext text/x-wiki stwbto6bt94hiqytxjjfxome6g4qalo 2448369 2448368 2026-08-17T15:18:13Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1176.eqiad.wmnet with OS bookworm 2448369 wikitext text/x-wiki snet8s35pfas65fhds2zx4gvbcvbvxe 2448370 2448369 2026-08-17T15:18:22Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1191.eqiad.wmnet with OS bookworm 2448370 wikitext text/x-wiki 4bhqjh3plztgn59fc74gu17p1lywde3 2448371 2448370 2026-08-17T15:18:37Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1190.eqiad.wmnet with OS bookworm 2448371 wikitext text/x-wiki ja0c0893qb3z4a9a55gbz2lh8qmvgr7 2448372 2448371 2026-08-17T15:19:03Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.reimage for host dns4004.wikimedia.org with OS trixie 2448372 wikitext text/x-wiki t8zs7v69agwus5eeaoxy0mcluo0gjd7 2448373 2448372 2026-08-17T15:19:20Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns1006.wikimedia.org with OS trixie 2448373 wikitext text/x-wiki 0pfk00dgsk0nsaxlcj5xp690bys3iip 2448374 2448373 2026-08-17T15:19:56Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1056~] - vriley@cumin1003" 2448374 wikitext text/x-wiki jadh51ff611kd1ymct88qh3rgwzmva1 2448375 2448374 2026-08-17T15:20:02Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1056~] - vriley@cumin1003" 2448375 wikitext text/x-wiki gzxhq1vfax5q4yaev1wpu527gol3vkj 2448376 2448375 2026-08-17T15:20:05Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448376 wikitext text/x-wiki tfuq5jceznrmbxw0qhkkpniobn495y0 2448377 2448376 2026-08-17T15:20:27Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1056 2448377 wikitext text/x-wiki 8hklprw9vkegee3i4gv6ubl9g2oviyu 2448378 2448377 2026-08-17T15:23:06Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.remove-downtime for dns1006.wikimedia.org 2448378 wikitext text/x-wiki crmpv54mljv8sa8lhho5bx5uv3tk4jw 2448379 2448378 2026-08-17T15:23:09Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for dns1006.wikimedia.org 2448379 wikitext text/x-wiki mqfzne4plg9j2585ir7hcskkv722651 2448380 2448379 2026-08-17T15:23:27Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns1006.wikimedia.org,service=authdns-update 2448380 wikitext text/x-wiki 88jiild73munk17519gdryy3kq7an6r 2448381 2448380 2026-08-17T15:23:35Z Stashbot 7414 sukhe@dns1004: START - running authdns-update 2448381 wikitext text/x-wiki 09lvbqhy7asv29bno0ecrqvizsxexju 2448382 2448381 2026-08-17T15:25:32Z Stashbot 7414 sukhe@dns1004: END - running authdns-update 2448382 wikitext text/x-wiki k0ljhhytqtcoz1hc8gnznalu6lg5534 2448383 2448382 2026-08-17T15:25:40Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1056 2448383 wikitext text/x-wiki 8tpey6z6voctbq0b4jz52k6q85rt0bw 2448384 2448383 2026-08-17T15:25:54Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns2006.wikimedia.org with OS trixie 2448384 wikitext text/x-wiki h71rzuyfx5matgdp7y9i6ohei1080bp 2448385 2448384 2026-08-17T15:26:23Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1056.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448385 wikitext text/x-wiki fe0ey1tpcn9vhq234h8m6p7bvpaxpsg 2448386 2448385 2026-08-17T15:26:40Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns1006.wikimedia.org 2448386 wikitext text/x-wiki sn2kkagaz6erszlz3bo9dqfghen2nf4 2448387 2448386 2026-08-17T15:26:56Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) rebooting A:liberica-ulsfo and A:liberica 2448387 wikitext text/x-wiki 039nh9pj5ic6vmp4rapnd6cup3kcqi9 2448388 2448387 2026-08-17T15:29:08Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.remove-downtime for dns2006.wikimedia.org 2448388 wikitext text/x-wiki qulwwb96gdwdv9rgjjpsvaavmslbqal 2448389 2448388 2026-08-17T15:29:11Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for dns2006.wikimedia.org 2448389 wikitext text/x-wiki kga92upc1tjyf1sbvm6wf9vi0v5at8s 2448390 2448389 2026-08-17T15:29:14Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns2006.wikimedia.org,service=authdns-update 2448390 wikitext text/x-wiki jbgd1qpsy47gjsqu7cs4ziy0tc338rq 2448391 2448390 2026-08-17T15:29:17Z Stashbot 7414 sukhe@dns1004: START - running authdns-update 2448391 wikitext text/x-wiki quixak8uqzcunecg50t70e34j1tughx 2448393 2448391 2026-08-17T15:31:12Z Stashbot 7414 sukhe@dns1004: END - running authdns-update 2448393 wikitext text/x-wiki 3ppe91jvl7b3tp000hl11u8zwmxqpog 2448394 2448393 2026-08-17T15:31:33Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns2006.wikimedia.org 2448394 wikitext text/x-wiki 75lbfvcbqdw2btbm1yn8bozn9yccud1 2448395 2448394 2026-08-17T15:33:08Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1190.eqiad.wmnet with reason: host reimage 2448395 wikitext text/x-wiki 7v2st7v68me7ok9hwenwfoetrhylym2 2448396 2448395 2026-08-17T15:33:42Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1191.eqiad.wmnet with reason: host reimage 2448396 wikitext text/x-wiki nxfp2muekyx4wpe4upjkp3hc965m8uc 2448397 2448396 2026-08-17T15:34:11Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1176.eqiad.wmnet with reason: host reimage 2448397 wikitext text/x-wiki gpd4go1cs78m4zgh676x6ngi4l1acbu 2448398 2448397 2026-08-17T15:34:33Z Stashbot 7414 moritzm: failover Ganeti master in eqiad to ganeti1046 2448398 wikitext text/x-wiki p92xj8re70p2fsepvgvcwcc7i81iwu9 2448399 2448398 2026-08-17T15:35:18Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1056.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448399 wikitext text/x-wiki 6j5pumy3k2f5eor9stklhvif9jlkmc9 2448401 2448399 2026-08-17T15:35:46Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1056.eqiad.wmnet with OS trixie 2448401 wikitext text/x-wiki 85gt51yrpxs4gc42lglj2qfvmevcqby 2448402 2448401 2026-08-17T15:35:52Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1054.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448402 wikitext text/x-wiki 9gval3ek9wzaiwno1yq5lp7lj2x22ce 2448403 2448402 2026-08-17T15:36:30Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1054.eqiad.wmnet with OS trixie 2448403 wikitext text/x-wiki 5tt7hewg9i00bbsvwep8ipvgq6nqi8p 2448405 2448403 2026-08-17T15:39:15Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1190.eqiad.wmnet with reason: host reimage 2448405 wikitext text/x-wiki e613o7q9nm6ytbsy6bc3f8l92bs6ai2 2448406 2448405 2026-08-17T15:42:54Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1191.eqiad.wmnet with reason: host reimage 2448406 wikitext text/x-wiki oiu5kg6f3jekuf1qmod35o2bphte1hd 2448407 2448406 2026-08-17T15:43:00Z Stashbot 7414 btullis@deploy1003: helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. 2448407 wikitext text/x-wiki as4dstlmxf37ge0ccxeb4hqrwrblkdm 2448408 2448407 2026-08-17T15:43:45Z Stashbot 7414 btullis@deploy1003: helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. 2448408 wikitext text/x-wiki izv2ldaib4zzne159maholpotlojogo 2448410 2448408 2026-08-17T15:45:35Z Stashbot 7414 btullis@deploy1003: helmfile [ml-serve-codfw] START helmfile.d/admin 'apply'. 2448410 wikitext text/x-wiki i1grfcwf7qo491llaunfp406vk28bbe 2448411 2448410 2026-08-17T15:46:16Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1176.eqiad.wmnet with reason: host reimage 2448411 wikitext text/x-wiki acbr7ll83vl5jbv2lqx1f1sahinvwn2 2448412 2448411 2026-08-17T15:46:59Z Stashbot 7414 btullis@deploy1003: helmfile [ml-serve-codfw] DONE helmfile.d/admin 'apply'. 2448412 wikitext text/x-wiki c34b67l0v1s6cngv43ng2m3b07wiv8y 2448413 2448412 2026-08-17T15:50:22Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on dns4004.wikimedia.org with reason: host reimage 2448413 wikitext text/x-wiki 9z8wp3uz5zv4eonfy2n9ocb4y7piaiv 2448414 2448413 2026-08-17T15:53:33Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns4004.wikimedia.org with reason: host reimage 2448414 wikitext text/x-wiki ouaqk25rstli3m4l4ekuh29n3tbo2q1 2448415 2448414 2026-08-17T15:55:29Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/mw-debug: apply 2448415 wikitext text/x-wiki l7g5cfwc8ylruwnwi7ps5h7x415euxn 2448416 2448415 2026-08-17T15:55:39Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-debug: apply 2448416 wikitext text/x-wiki 6k7bsjnx2ffvtbytvidqhdfya1ur3kn 2448417 2448416 2026-08-17T15:55:49Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-debug: apply 2448417 wikitext text/x-wiki iws0129dyvkekt1jkjulaqk6yxnzx67 2448418 2448417 2026-08-17T15:55:56Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply 2448418 wikitext text/x-wiki gswjxswdi8zoma1d1twoullcg951msz 2448419 2448418 2026-08-17T15:59:09Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1154.eqiad.wmnet with reason: host reimage 2448419 wikitext text/x-wiki aem79fynz1vfui5izwzvqekul7u6nve 2448420 2448419 2026-08-17T15:59:48Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1190.eqiad.wmnet with OS bookworm 2448420 wikitext text/x-wiki ifmqfkwx23yw2rimccv6a2traqvpqgj 2448421 2448420 2026-08-17T16:03:16Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching aqs[2002-2012].codfw.wmnet,aqs[1017-1027].eqiad.wmnet: Upgrade Cassandra to 5.0.8 — T433026 - eevans@cumin1003 2448421 wikitext text/x-wiki 8j7xfbydayvdpmlbn8mkvej1j87ood0 2448422 2448421 2026-08-17T16:03:47Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1154.eqiad.wmnet with reason: host reimage 2448422 wikitext text/x-wiki bbj4gh34sppoyzqr5by49txlpljf3l6 2448423 2448422 2026-08-17T16:05:25Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1191.eqiad.wmnet with OS bookworm 2448423 wikitext text/x-wiki 78rxmr3cvzpnt5bwtp5p0gcxozz6wr2 2448424 2448423 2026-08-17T16:06:24Z Stashbot 7414 btullis@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. 2448424 wikitext text/x-wiki l1ypse2mualdkffqdnhxx8u3kzkmbiu 2448425 2448424 2026-08-17T16:07:51Z Stashbot 7414 btullis@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. 2448425 wikitext text/x-wiki heponjt2tr8jz8nlq42zkubn16z59et 2448426 2448425 2026-08-17T16:09:09Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1176.eqiad.wmnet with OS bookworm 2448426 wikitext text/x-wiki 9c6pnjs7cjhercfmy6prjmor59tt7p1 2448427 2448426 2026-08-17T16:09:58Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] START helmfile.d/services/eventgate-analytics-external: apply 2448427 wikitext text/x-wiki t2w6p6tvyt4x6aljxxag2nmd9ocq66b 2448428 2448427 2026-08-17T16:11:02Z Stashbot 7414 phuedx@deploy1003: helmfile [staging] DONE helmfile.d/services/eventgate-analytics-external: apply 2448428 wikitext text/x-wiki 08rfq39vg5htl61anofpaivosbn3t2c 2448429 2448428 2026-08-17T16:12:39Z Stashbot 7414 phuedx@deploy1003: helmfile [codfw] START helmfile.d/services/eventgate-analytics-external: apply 2448429 wikitext text/x-wiki jss9x1h2ovnqel0wh92u78s0l22wy7f 2448430 2448429 2026-08-17T16:13:45Z Stashbot 7414 phuedx@deploy1003: helmfile [codfw] DONE helmfile.d/services/eventgate-analytics-external: apply 2448430 wikitext text/x-wiki qni4ji0nvuelaf1wfwhvl290svxkn2m 2448431 2448430 2026-08-17T16:15:03Z Stashbot 7414 phuedx@deploy1003: helmfile [eqiad] START helmfile.d/services/eventgate-analytics-external: apply 2448431 wikitext text/x-wiki edhrdx0venmfumr8bv2yk5j2ps2h5eg 2448432 2448431 2026-08-17T16:16:05Z Stashbot 7414 phuedx@deploy1003: helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics-external: apply 2448432 wikitext text/x-wiki 24wcapx4worsntcyb2y05qk5s9ou4ut 2448433 2448432 2026-08-17T16:17:40Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2448433 wikitext text/x-wiki 3u6yj2uwlazwqzvol0jc7bwj1zgetxg 2448434 2448433 2026-08-17T16:19:07Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.admin rebooting A:liberica-drmrs and A:liberica 2448434 wikitext text/x-wiki p6om7re0c3rnl20fn2c960tyun3k758 2448435 2448434 2026-08-17T16:21:46Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1057] - vriley@cumin1003" 2448435 wikitext text/x-wiki 0ksebe2uduligw9iresaa5q5l7r57ew 2448436 2448435 2026-08-17T16:21:51Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt [cloudvirt1057] - vriley@cumin1003" 2448436 wikitext text/x-wiki 13jrfmulg0kmlcx3toq0kfqzkbxgrm4 2448437 2448436 2026-08-17T16:21:53Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448437 wikitext text/x-wiki 7ytpv8o386j5ccnunsrs69jnayy7yaq 2448438 2448437 2026-08-17T16:22:08Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1057 2448438 wikitext text/x-wiki lb1kkmyljwghi6mtebrvcil32xt8rxf 2448439 2448438 2026-08-17T16:22:23Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1057 2448439 wikitext text/x-wiki 5wu4mf9j03uhw6vwd0yivecruw8fahi 2448440 2448439 2026-08-17T16:23:43Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1055.eqiad.wmnet with OS trixie 2448440 wikitext text/x-wiki beql9ocad184vivgened5cz4mpc8qtg 2448441 2448440 2026-08-17T16:23:57Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host cloudvirt1057.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448441 wikitext text/x-wiki 3ikxgm0gh0nfqgbp68xpi0qxfyaiwe3 2448442 2448441 2026-08-17T16:29:00Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1154.eqiad.wmnet with OS bookworm 2448442 wikitext text/x-wiki 8m1tznz0kx4phdjk7vscp4bs55bmzxr 2448443 2448442 2026-08-17T16:29:38Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) rebooting A:liberica-drmrs and A:liberica 2448443 wikitext text/x-wiki cck4tqafsewao8uk8vqfyuot6ff9raj 2448445 2448443 2026-08-17T16:30:58Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudvirt1057.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2448445 wikitext text/x-wiki 9aque6wztukbtbancwn44pdvt8vrle9 2448446 2448445 2026-08-17T16:31:28Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1057.eqiad.wmnet with OS trixie 2448446 wikitext text/x-wiki kxe14mnmhkan7kj2vt3k93l5t70hehy 2448447 2448446 2026-08-17T16:33:31Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1157.eqiad.wmnet with OS bookworm 2448447 wikitext text/x-wiki nd65xjdfewjs0zfh3i2wqsal1j74f4h 2448448 2448447 2026-08-17T16:33:34Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1158.eqiad.wmnet with OS bookworm 2448448 wikitext text/x-wiki jg491sg26d9hzsd2wg16zkojsczltjn 2448449 2448448 2026-08-17T16:33:37Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1159.eqiad.wmnet with OS bookworm 2448449 wikitext text/x-wiki s5b1kg0oarv8pmja5x4ivvdtol3euv4 2448450 2448449 2026-08-17T16:33:42Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1183.eqiad.wmnet with OS bookworm 2448450 wikitext text/x-wiki dcqgayp9zhcrzo3zff5ptwi8x10xb0l 2448451 2448450 2026-08-17T16:33:45Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1192.eqiad.wmnet with OS bookworm 2448451 wikitext text/x-wiki 0x7zaut9do3ami3zn2tlbfsweraqzaq 2448454 2448451 2026-08-17T16:39:06Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1180.eqiad.wmnet with OS bookworm 2448454 wikitext text/x-wiki 0xt925xuyo7fqcqeo9ddapba077gwhm 2448455 2448454 2026-08-17T16:39:22Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1193.eqiad.wmnet with OS bookworm 2448455 wikitext text/x-wiki n2aybp05ha3lszula8dikelurbad4ng 2448456 2448455 2026-08-17T16:39:32Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.move-vlan for host an-worker1180 2448456 wikitext text/x-wiki 5jswfzdyexxy1ah7tqqyr0zbt0dr8bm 2448457 2448456 2026-08-17T16:39:45Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.dns.netbox 2448457 wikitext text/x-wiki jzwik7ekz042g0qx2lh3ws0ffae1xsn 2448458 2448457 2026-08-17T16:43:37Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns4004.wikimedia.org with OS trixie 2448458 wikitext text/x-wiki a54htzbpqjazakacunv8go1odh5hao8 2448459 2448458 2026-08-17T16:44:10Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns4004.wikimedia.org,service=authdns-update 2448459 wikitext text/x-wiki cpabb11ts6ujnhwmgxmy73ncfz82haz 2448460 2448459 2026-08-17T16:44:15Z Stashbot 7414 sukhe@dns1004: START - running authdns-update 2448460 wikitext text/x-wiki 1dkdhn76gxfm6nrx2gpvonvoe0938po 2448461 2448460 2026-08-17T16:46:11Z Stashbot 7414 sukhe@dns1004: END - running authdns-update 2448461 wikitext text/x-wiki pxcxlt50la46qjxte7v4xnhc6m7fw3s 2448462 2448461 2026-08-17T16:46:29Z Stashbot 7414 sukhe@puppetserver1001: conftool action : set/pooled=yes; selector: name=dns4004.wikimedia.org 2448462 wikitext text/x-wiki 9zjoef035ytn2t7xyqubybbypjcv8nx 2448463 2448462 2026-08-17T16:48:00Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1192.eqiad.wmnet with reason: host reimage 2448463 wikitext text/x-wiki bzobgy0bnjoz6c55r11wzydxaoken3u 2448464 2448463 2026-08-17T16:48:49Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1159.eqiad.wmnet with reason: host reimage 2448464 wikitext text/x-wiki hofanmj2jcra823tchw2plvbx2626ut 2448465 2448464 2026-08-17T16:49:19Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1183.eqiad.wmnet with reason: host reimage 2448465 wikitext text/x-wiki 08r84bxs9p5ph0yjxouwvi3ide1ciey 2448467 2448465 2026-08-17T16:49:35Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.loadbalancer.admin rebooting A:liberica-eqsin and A:liberica 2448467 wikitext text/x-wiki c0pj8qlyrgfdbn1vla6avc0of4mbzso 2448468 2448467 2026-08-17T16:49:40Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1157.eqiad.wmnet with reason: host reimage 2448468 wikitext text/x-wiki 677irpob00h3xk6yz7q5k9bvkrpg14a 2448469 2448468 2026-08-17T16:49:50Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1158.eqiad.wmnet with reason: host reimage 2448469 wikitext text/x-wiki i4nzd1hvlb43r5vj5ngr9uzkzmrv9fg 2448470 2448469 2026-08-17T16:50:26Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host an-worker1180 - btullis@cumin1003" 2448470 wikitext text/x-wiki lcghi57p661epau2qgfx52buag15fj6 2448471 2448470 2026-08-17T16:50:31Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host an-worker1180 - btullis@cumin1003" 2448471 wikitext text/x-wiki saktzv464ge9k6vxl8dkjfkb4sujyg2 2448472 2448471 2026-08-17T16:50:34Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448472 wikitext text/x-wiki 4cnjnflpzolabu4o8kqrlhdueref49i 2448473 2448472 2026-08-17T16:50:37Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.dns.wipe-cache an-worker1180.eqiad.wmnet 17.36.64.10.in-addr.arpa 7.1.0.0.6.3.0.0.4.6.0.0.0.1.0.0.6.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2448473 wikitext text/x-wiki 58fikow7nzh6ehpbyn4omlww6po5uuo 2448474 2448473 2026-08-17T16:50:39Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) an-worker1180.eqiad.wmnet 17.36.64.10.in-addr.arpa 7.1.0.0.6.3.0.0.4.6.0.0.0.1.0.0.6.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2448474 wikitext text/x-wiki scz3ajvnfifeacgseaitt81qe1vk4vt 2448475 2448474 2026-08-17T16:50:41Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host an-worker1180 2448475 wikitext text/x-wiki 4w6s4v8s6wjk4zzb6fu8j16cdr35lcl 2448476 2448475 2026-08-17T16:51:31Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.cassandra.roll-restart for nodes matching A:aqs-eqiad: Upgrade to Java 17 — T433026 - eevans@cumin1003 2448476 wikitext text/x-wiki qjlaq32sqypt45rxeowvbtp5ct2nkh5 2448477 2448476 2026-08-17T16:54:01Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1192.eqiad.wmnet with reason: host reimage 2448477 wikitext text/x-wiki 0tiqc8o80o49qagzkeqetbf8idg020k 2448478 2448477 2026-08-17T16:54:48Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1193.eqiad.wmnet with reason: host reimage 2448478 wikitext text/x-wiki 401ef2vqvgdnly1014xj2maj3x2p44j 2448479 2448478 2026-08-17T16:56:00Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1056.eqiad.wmnet with OS trixie 2448479 wikitext text/x-wiki dv8i7rtmhw8ifmrsxat0ncaahr3odf9 2448480 2448479 2026-08-17T16:56:45Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1054.eqiad.wmnet with OS trixie 2448480 wikitext text/x-wiki 7wwbbg749d78dn45k2kmuyec6ok0han 2448481 2448480 2026-08-17T16:57:12Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1183.eqiad.wmnet with reason: host reimage 2448481 wikitext text/x-wiki rhynhqhb86i6crlbxwlpcyi71i7xqqe 2448483 2448481 2026-08-17T17:01:02Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1193.eqiad.wmnet with reason: host reimage 2448483 wikitext text/x-wiki ag9rwjrg4iduqen6hs3njvgz0l6gcfa 2448484 2448483 2026-08-17T17:01:16Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) rebooting A:liberica-eqsin and A:liberica 2448484 wikitext text/x-wiki 3xi8vg7pmeb260q8zhgfwt9n1egmphy 2448485 2448484 2026-08-17T17:02:43Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host an-worker1180 2448485 wikitext text/x-wiki nve0dopm51vbtoxujdpathxmbaaaeh1 2448486 2448485 2026-08-17T17:02:46Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host an-worker1180 2448486 wikitext text/x-wiki oujyh3ycvdjre9xhidqfqri83kqzrie 2448487 2448486 2026-08-17T17:04:49Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1157.eqiad.wmnet with reason: host reimage 2448487 wikitext text/x-wiki ngc4epwl7k6ql3p5rc9woo2kpah1n2v 2448489 2448487 2026-08-17T17:09:46Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1158.eqiad.wmnet with reason: host reimage 2448489 wikitext text/x-wiki bfbhjjyp1e9x9o6ylhjae6wt1fw82dv 2448490 2448489 2026-08-17T17:12:42Z Stashbot 7414 swfrench@deploy1003: helmfile [staging-codfw] START helmfile.d/admin 'apply'. 2448490 wikitext text/x-wiki ihgz2i4glwjc35o13p2pil3dgy0nr7a 2448491 2448490 2026-08-17T17:13:57Z Stashbot 7414 swfrench@deploy1003: helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. 2448491 wikitext text/x-wiki 88zqicsmaayz4fx914chkkptuvs0nd8 2448492 2448491 2026-08-17T17:14:28Z Stashbot 7414 swfrench@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2448492 wikitext text/x-wiki m62nrxzpwx4aizafjqy4gl0n43fxzuu 2448493 2448492 2026-08-17T17:14:42Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1159.eqiad.wmnet with reason: host reimage 2448493 wikitext text/x-wiki sp6vg4spc6tmf6jkrvk7vnrdq2thbpr 2448494 2448493 2026-08-17T17:15:10Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1192.eqiad.wmnet with OS bookworm 2448494 wikitext text/x-wiki cowak8zcz21jc9th79rqbtt0wcnbwud 2448495 2448494 2026-08-17T17:15:19Z Stashbot 7414 swfrench@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2448495 wikitext text/x-wiki a05975izprzlzcr6ie9fn48ukpu6u1k 2448496 2448495 2026-08-17T17:15:45Z Stashbot 7414 swfrench@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2448496 wikitext text/x-wiki e60zrr5zlbsps95pbh17fmdfv7hhqwq 2448497 2448496 2026-08-17T17:16:10Z Stashbot 7414 swfrench@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2448497 wikitext text/x-wiki qqrne6xnfhr6mz0jvl46oh6x6otbqej 2448498 2448497 2026-08-17T17:16:29Z Stashbot 7414 swfrench@deploy1003: helmfile [staging-eqiad] START helmfile.d/admin 'apply'. 2448498 wikitext text/x-wiki n6vj7c1qug1qa108g27k9au6l1dyfqd 2448499 2448498 2026-08-17T17:17:07Z Stashbot 7414 swfrench@deploy1003: helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. 2448499 wikitext text/x-wiki hi6218f3qexx7owjk3z2shxk6ndlmxp 2448500 2448499 2026-08-17T17:17:32Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2448500 wikitext text/x-wiki i9xljnjhm693c15st88gu5fgp7hqlde 2448501 2448500 2026-08-17T17:17:49Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1180.eqiad.wmnet with reason: host reimage 2448501 wikitext text/x-wiki lcl5l9yyhkleyfv5yfwex2bnoxc9rvz 2448502 2448501 2026-08-17T17:18:18Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2448502 wikitext text/x-wiki 4ph2azm11ce968hnievpd1irm8qmv4y 2448503 2448502 2026-08-17T17:19:00Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] START helmfile.d/admin 'apply'. 2448503 wikitext text/x-wiki 9bgsji2kfwdmfq1o1hmgefgzvlx5881 2448504 2448503 2026-08-17T17:20:06Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1183.eqiad.wmnet with OS bookworm 2448504 wikitext text/x-wiki pc7pyc0j5x077qjtmbxw8xgs1ot2bkc 2448505 2448504 2026-08-17T17:20:24Z Stashbot 7414 swfrench@deploy1003: helmfile [codfw] DONE helmfile.d/admin 'apply'. 2448505 wikitext text/x-wiki orfzhb44qrizc8bu0618yfqpg7czel9 2448506 2448505 2026-08-17T17:20:48Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2448506 wikitext text/x-wiki 5mm7tpct4fbxnufvovd5fpbuiu24stz 2448507 2448506 2026-08-17T17:21:11Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2448507 wikitext text/x-wiki 2lxgzvjr12wuyzudq134ycyp0zjcz1t 2448508 2448507 2026-08-17T17:21:38Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] START helmfile.d/admin 'apply'. 2448508 wikitext text/x-wiki 4dej52491rbw2zvcs2eizywt69dwayy 2448509 2448508 2026-08-17T17:21:41Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1193.eqiad.wmnet with OS bookworm 2448509 wikitext text/x-wiki 01x128oy08m72kqzgq7zfafytiqtt7z 2448510 2448509 2026-08-17T17:22:17Z Stashbot 7414 swfrench@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'apply'. 2448510 wikitext text/x-wiki i475mxisaobnxcna8bn0y0x0p1x39o3 2448511 2448510 2026-08-17T17:23:17Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1180.eqiad.wmnet with reason: host reimage 2448511 wikitext text/x-wiki 60gr4ni43enhfw1ltt4v64h6vxhawn4 2448512 2448511 2026-08-17T17:28:35Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1157.eqiad.wmnet with OS bookworm 2448512 wikitext text/x-wiki b6vln1xvwb95xa5ge6k0qczgpce3ji1 2448513 2448512 2026-08-17T17:30:25Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on A:cp-ulsfo or A:cp-drmrs and A:cp - 9.2.15 upgrade (T434620) 2448513 wikitext text/x-wiki p7boi0mjfpy96amdpodwl0mw3nkh9zn 2448514 2448513 2026-08-17T17:30:58Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:aqs-eqiad: Upgrade to Java 17 — T433026 - eevans@cumin1003 2448514 wikitext text/x-wiki f2mmwpfibjobuijdcn3eiavvkgdun9x 2448515 2448514 2026-08-17T17:34:14Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1158.eqiad.wmnet with OS bookworm 2448515 wikitext text/x-wiki d9hyhmv9ksbzjj2yyby6pwzjxezspwm 2448517 2448515 2026-08-17T17:36:37Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1159.eqiad.wmnet with OS bookworm 2448517 wikitext text/x-wiki 2h6cerf4rte79u034ktate492f4vy5a 2448519 2448517 2026-08-17T17:38:49Z Stashbot 7414 swfrench@deploy1003: Started scap sync-world: Deploy "Point Test Wiki to new docroot" - T432412 2448519 wikitext text/x-wiki rj4h4bv98o5t7aljb6fuayvozmbjaim 2448520 2448519 2026-08-17T17:39:09Z Stashbot 7414 cdobbins@cumin1003: START - Cookbook sre.cdn.roll-upgrade-ats Rolling upgrade of ATS on A:cp-eqsin and A:cp - 9.2.15 upgrade (T434478) 2448520 wikitext text/x-wiki f4u350wz0akg714q4v7kzu5jknw399m 2448521 2448520 2026-08-17T17:39:44Z Stashbot 7414 swfrench@deploy1003: swfrench: Deploy "Point Test Wiki to new docroot" - T432412 synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448521 wikitext text/x-wiki 87vjsjba1vlllz5idgrx8trnz0f0itq 2448523 2448521 2026-08-17T17:47:50Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1180.eqiad.wmnet with OS bookworm 2448523 wikitext text/x-wiki 82ri6vsjlojzkw6fkhd5j04nwge0hkr 2448524 2448523 2026-08-17T17:51:43Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1057.eqiad.wmnet with OS trixie 2448524 wikitext text/x-wiki i3d34dp66kyv9jqbq898ssbhqbgeblo 2448526 2448524 2026-08-17T18:00:44Z Stashbot 7414 swfrench@deploy1003: swfrench: Continuing with deployment 2448526 wikitext text/x-wiki o8olvhi542aqcvwc4bp5i5nwmjn5g6g 2448528 2448526 2026-08-17T18:04:53Z Stashbot 7414 swfrench@deploy1003: Finished scap sync-world: Deploy "Point Test Wiki to new docroot" - T432412 (duration: 26m 03s) 2448528 wikitext text/x-wiki oegm2cku0b8pszul6jn5otp99eb7xw4 2448530 2448528 2026-08-17T18:14:05Z Stashbot 7414 dwisehaupt@dns1005: START - running authdns-update 2448530 wikitext text/x-wiki 44iprfh7r585mi9llx2cu8ujrxcc6mk 2448532 2448530 2026-08-17T18:16:10Z Stashbot 7414 dwisehaupt@dns1005: END - running authdns-update 2448532 wikitext text/x-wiki buxenfm38mniz165t0e13jaq5k52l4j 2448533 2448532 2026-08-17T18:23:50Z Stashbot 7414 swfrench@cumin2002: START - Cookbook sre.hosts.reboot-single for host conf2007.codfw.wmnet 2448533 wikitext text/x-wiki nb0k247bfd3ka2x4v4eyzy86l08xjhe 2448534 2448533 2026-08-17T18:30:06Z Stashbot 7414 swfrench@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host conf2007.codfw.wmnet 2448534 wikitext text/x-wiki 3ap4h16s2dasw9xypeeeseq355j1czf 2448536 2448534 2026-08-17T18:34:44Z Stashbot 7414 swfrench@cumin2002: START - Cookbook sre.hosts.reboot-single for host conf2008.codfw.wmnet 2448536 wikitext text/x-wiki nyqnpl86x5k91sivgcnsmyfb4rcjm2z 2448538 2448536 2026-08-17T18:41:12Z Stashbot 7414 swfrench@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host conf2008.codfw.wmnet 2448538 wikitext text/x-wiki rpci5smw50v6sbr7ipa2569g4i81zzz 2448539 2448538 2026-08-17T18:47:25Z Stashbot 7414 swfrench@cumin2002: START - Cookbook sre.hosts.reboot-single for host conf2009.codfw.wmnet 2448539 wikitext text/x-wiki s9qcz92g2tqsea42bvduqibb5uro4tz 2448542 2448539 2026-08-17T18:52:46Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.281.1" for 3 host(s) 2448542 wikitext text/x-wiki sylhkss87fvdzlfxubh73yyfsat2ve4 2448543 2448542 2026-08-17T18:53:08Z Stashbot 7414 dwisehaupt@dns1006: START - running authdns-update 2448543 wikitext text/x-wiki sch76gkv3k9vig7sgynriyvo8rkj5bc 2448544 2448543 2026-08-17T18:54:06Z Stashbot 7414 swfrench@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host conf2009.codfw.wmnet 2448544 wikitext text/x-wiki 6xt8i7he62rl0kf9yx5n09kd6di92qr 2448546 2448544 2026-08-17T18:54:41Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.281.1" completed for 3 hosts 2448546 wikitext text/x-wiki st8qjknarkxbwjq8ipbzdru35qtw6r5 2448547 2448546 2026-08-17T18:55:17Z Stashbot 7414 dwisehaupt@dns1006: END - running authdns-update 2448547 wikitext text/x-wiki kmhwqyo989lf1aubt1p43dsha0g0lm2 2448548 2448547 2026-08-17T18:55:27Z Stashbot 7414 dancy@deploy1003: Started scap sync-world: testing T375514 2448548 wikitext text/x-wiki 0an068tp69yzowq5g6rxtaibrm42rgv 2448549 2448548 2026-08-17T18:58:40Z Stashbot 7414 dancy@deploy1003: Finished scap sync-world: testing T375514 (duration: 03m 13s) 2448549 wikitext text/x-wiki ru0k7xf9j61k0mx9nkppdr75t7h9x2s 2448550 2448549 2026-08-17T19:03:21Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.cassandra.roll-restart for nodes matching A:aqs-codfw: Upgrade to Java 17 — T433026 - eevans@cumin1003 2448550 wikitext text/x-wiki 4ocnxb1mui9g0ybvctkvthcp3looez8 2448551 2448550 2026-08-17T19:09:34Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2448551 wikitext text/x-wiki soasf76l4a1p8562ujhtkaq71ho1gzo 2448552 2448551 2026-08-17T19:10:08Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2448552 wikitext text/x-wiki 59wl7pkklfbddwwklwkfw0xar8cln3f 2448553 2448552 2026-08-17T19:10:40Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1160.eqiad.wmnet with OS bookworm 2448553 wikitext text/x-wiki 807xfjm0e8wlgjst3fwm2vdjl5fmxj1 2448554 2448553 2026-08-17T19:10:43Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1161.eqiad.wmnet with OS bookworm 2448554 wikitext text/x-wiki tdusl7utwo36j78lxa2pybobjknub9q 2448555 2448554 2026-08-17T19:10:46Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1162.eqiad.wmnet with OS bookworm 2448555 wikitext text/x-wiki 8l4jsc8jm5iq6krg1x0mrpk4i7c0y5b 2448556 2448555 2026-08-17T19:10:50Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1184.eqiad.wmnet with OS bookworm 2448556 wikitext text/x-wiki kdsmdda4ygzt4x8n36g6lih5um2zprl 2448557 2448556 2026-08-17T19:10:53Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1194.eqiad.wmnet with OS bookworm 2448557 wikitext text/x-wiki kihstrnnyaetimkxoee1s412igekjyb 2448558 2448557 2026-08-17T19:11:03Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1195.eqiad.wmnet with OS bookworm 2448558 wikitext text/x-wiki 5gph74xt81ki7p87bluo52wgsajw09z 2448559 2448558 2026-08-17T19:19:27Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-test: apply 2448559 wikitext text/x-wiki a3srf5xzgozmrr5ry61dvb2abm4c43j 2448560 2448559 2026-08-17T19:25:12Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1195.eqiad.wmnet with reason: host reimage 2448560 wikitext text/x-wiki 5paj6g7i6xq2v3xnk4evz3lmsexb27s 2448561 2448560 2026-08-17T19:26:02Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1162.eqiad.wmnet with reason: host reimage 2448561 wikitext text/x-wiki 9k2htihpkwecamsspo9iji2h1c1pvpu 2448562 2448561 2026-08-17T19:26:19Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1184.eqiad.wmnet with reason: host reimage 2448562 wikitext text/x-wiki 3un2qfpt3hew6u892rvbz4y2lilvc2u 2448563 2448562 2026-08-17T19:26:27Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1160.eqiad.wmnet with reason: host reimage 2448563 wikitext text/x-wiki co7eealsjj188tewheeqwmu5rrnumc8 2448564 2448563 2026-08-17T19:26:42Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1161.eqiad.wmnet with reason: host reimage 2448564 wikitext text/x-wiki l55pf1h2wec8dxkgdaos4ucylrc8av1 2448565 2448564 2026-08-17T19:29:16Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1195.eqiad.wmnet with reason: host reimage 2448565 wikitext text/x-wiki ou8zisyfbpbt00hb2z71b8l7ggdlgyw 2448567 2448565 2026-08-17T19:32:55Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1161.eqiad.wmnet with reason: host reimage 2448567 wikitext text/x-wiki 6vvq9xibicbp9dplvmynudc6fd9uodu 2448574 2448567 2026-08-17T19:36:19Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1184.eqiad.wmnet with reason: host reimage 2448574 wikitext text/x-wiki gq0b473hwp3dee7gmloxaw3yjax2zua 2448576 2448574 2026-08-17T19:36:29Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-test: apply 2448576 wikitext text/x-wiki 9rw4500l1rg0uqk758q2l3pkq94ouvv 2448577 2448576 2026-08-17T19:36:31Z Stashbot 7414 cdobbins@cumin1003: END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on A:cp-eqsin and A:cp - 9.2.15 upgrade (T434478) 2448577 wikitext text/x-wiki kyh9mg9b90bw7moklr4m7s3qan5hen0 2448579 2448577 2026-08-17T19:39:41Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1162.eqiad.wmnet with reason: host reimage 2448579 wikitext text/x-wiki ti3gwejtm8h2dx0aejksijjk5yx28fi 2448580 2448579 2026-08-17T19:41:37Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:aqs-codfw: Upgrade to Java 17 — T433026 - eevans@cumin1003 2448580 wikitext text/x-wiki 08ydshqhrcxvwimih47vlds8mqmzov4 2448581 2448580 2026-08-17T19:44:31Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1160.eqiad.wmnet with reason: host reimage 2448581 wikitext text/x-wiki f27vgvrpuqvqs25uukfp9euwn6td06g 2448582 2448581 2026-08-17T19:49:41Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-test: apply 2448582 wikitext text/x-wiki o2oxv9bvqnvyhl2o9m3h1fx4pdjg27y 2448583 2448582 2026-08-17T19:49:44Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-test: apply 2448583 wikitext text/x-wiki e448s5oo2y6zg59ab841v8lp1624lyk 2448584 2448583 2026-08-17T19:49:57Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1195.eqiad.wmnet with OS bookworm 2448584 wikitext text/x-wiki 1corkcgev4yi7fng3u6kr7vuu4xtiz8 2448585 2448584 2026-08-17T19:55:20Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1161.eqiad.wmnet with OS bookworm 2448585 wikitext text/x-wiki my8wckmreqbixunqr8fntucu54ebikk 2448586 2448585 2026-08-17T19:58:44Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1184.eqiad.wmnet with OS bookworm 2448586 wikitext text/x-wiki 2q98ijy0gvx1bm8db1rsokoex0oajrn 2448588 2448586 2026-08-17T20:02:49Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1162.eqiad.wmnet with OS bookworm 2448588 wikitext text/x-wiki rj7u4ucafyaapaqhlqvdw9sy7vodhna 2448590 2448588 2026-08-17T20:06:48Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1160.eqiad.wmnet with OS bookworm 2448590 wikitext text/x-wiki fb59lzvrkoforxseuitz2eyabt4dv32 2448591 2448590 2026-08-17T20:08:55Z Stashbot 7414 cjming@deploy1003: Started scap sync-world: Backport for [[gerrit:1326077|InstrumentConstructiveEdits: anchor all runs to the nearest `interval` (T431493)]] 2448591 wikitext text/x-wiki ewqzgh8g6l0mthbezomsg8pg0l0q2nb 2448592 2448591 2026-08-17T20:10:48Z Stashbot 7414 cjming@deploy1003: cjming: Backport for [[gerrit:1326077|InstrumentConstructiveEdits: anchor all runs to the nearest `interval` (T431493)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448592 wikitext text/x-wiki 1vv6v7t3pi1f59mo9vzocw8pnoqsmml 2448593 2448592 2026-08-17T20:11:06Z Stashbot 7414 cjming@deploy1003: cjming: Continuing with deployment 2448593 wikitext text/x-wiki 702kvg6gi9qlbo3fzi7jp54kd72bpb3 2448594 2448593 2026-08-17T20:15:21Z Stashbot 7414 cjming@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326077|InstrumentConstructiveEdits: anchor all runs to the nearest `interval` (T431493)]] (duration: 06m 25s) 2448594 wikitext text/x-wiki 3pmn57snj656h25r8su1al863mdbsxm 2448595 2448594 2026-08-17T20:17:24Z Stashbot 7414 cjming@deploy1003: Started scap sync-world: Backport for [[gerrit:1326361|Remove escaped paths in app site association file (T432412)]] 2448595 wikitext text/x-wiki pzt5kcdb8kwq0cg1macqg0uwjyd28zu 2448596 2448595 2026-08-17T20:19:14Z Stashbot 7414 cjming@deploy1003: cjming, tsev: Backport for [[gerrit:1326361|Remove escaped paths in app site association file (T432412)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448596 wikitext text/x-wiki ht7ngba24909crlnregu89w7syvgufl 2448597 2448596 2026-08-17T20:21:47Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.cassandra.roll-restart for nodes matching A:aqs-eqiad: Set storage compatability to UPGRADING — T433026 - eevans@cumin1003 2448597 wikitext text/x-wiki qpw3u0jorx66qtcp4840vpsvbrwtrms 2448601 2448597 2026-08-17T20:24:40Z Stashbot 7414 inflatador: bking@deploy1003 `charlie --services_dir dse-k8s-services -s opensearch-* -e dse-k8s-*` T435125 2448601 wikitext text/x-wiki 71cn6y19w0nmgssxvj7pkrva8mzyapb 2448602 2448601 2026-08-17T20:26:35Z Stashbot 7414 cjming@deploy1003: cjming, tsev: Continuing with deployment 2448602 wikitext text/x-wiki jnonq8q333ibspj4ny9jm2mos037dw1 2448603 2448602 2026-08-17T20:26:50Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-apifeatureusage: apply 2448603 wikitext text/x-wiki 4e8hvqe06i29tgbpujgog6z2fdbocyy 2448604 2448603 2026-08-17T20:26:53Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-apifeatureusage: apply 2448604 wikitext text/x-wiki 6a7ecdu5n7ndgs3f0c2bbuml3sox183 2448605 2448604 2026-08-17T20:27:00Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-apifeatureusage: apply 2448605 wikitext text/x-wiki 4bmj7hecs884n3hxcf3g4ufg1bvrwz1 2448606 2448605 2026-08-17T20:27:06Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-apifeatureusage: apply 2448606 wikitext text/x-wiki l5v2myfsgl836g8q9vcwuct7lsqfm5o 2448607 2448606 2026-08-17T20:27:17Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-apifeatureusage-test: apply 2448607 wikitext text/x-wiki 4k9mq7w9129lewosg40s394ogxn8cij 2448608 2448607 2026-08-17T20:27:20Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-apifeatureusage-test: apply 2448608 wikitext text/x-wiki 3v7uqe1s7d6woanywd6w28t9lh0fru0 2448609 2448608 2026-08-17T20:27:31Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-apifeatureusage-test: apply 2448609 wikitext text/x-wiki pz6dz1417mlmz87gm7p6io8394k5n3i 2448610 2448609 2026-08-17T20:27:36Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-apifeatureusage-test: apply 2448610 wikitext text/x-wiki a55ycmtbwvjgzhi07mhrvppbx5guzvh 2448611 2448610 2026-08-17T20:27:42Z Stashbot 7414 inflatador: bking@deploy1003 `charlie --services_dir dse-k8s-services -s opensearch-* -e dse-k8s-* apply` T435125 2448611 wikitext text/x-wiki tka5zxmln5nji6igtiplzfuer1rrqto 2448612 2448611 2026-08-17T20:28:06Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply 2448612 wikitext text/x-wiki qtbxm8ibxm2aeq10ika29tg8bzul28v 2448613 2448612 2026-08-17T20:28:10Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply 2448613 wikitext text/x-wiki 89gmx6wsktunoqpbokssadfsot7sw5s 2448614 2448613 2026-08-17T20:28:15Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply 2448614 wikitext text/x-wiki h17imqjq8u8dgso7jy3xv7jex9eirtn 2448615 2448614 2026-08-17T20:28:21Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply 2448615 wikitext text/x-wiki 5bqeslk0fu3h1ybalbb47ev3265mnr3 2448616 2448615 2026-08-17T20:28:25Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search: apply 2448616 wikitext text/x-wiki hg7rt4xictxw6n6aunirslf4v47nlur 2448617 2448616 2026-08-17T20:28:29Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search: apply 2448617 wikitext text/x-wiki axz5vg0cnv77efc5ygg13ks1nhcrsto 2448618 2448617 2026-08-17T20:28:42Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-semantic-search: apply 2448618 wikitext text/x-wiki ll51mntdhnpphb5w59h7bvqsku3ycff 2448619 2448618 2026-08-17T20:28:49Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search: apply 2448619 wikitext text/x-wiki lnk7lg3rjywnv1m2sgmr1ak0bj1wozp 2448620 2448619 2026-08-17T20:29:01Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-semantic-search-test: apply 2448620 wikitext text/x-wiki eelnqna0s4l0a1lobcdq9bbuspv1cb0 2448621 2448620 2026-08-17T20:29:05Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-test: apply 2448621 wikitext text/x-wiki ihe58n30l94vkxfvqr9c2jvj591esvw 2448622 2448621 2026-08-17T20:29:10Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-semantic-search-test: apply 2448622 wikitext text/x-wiki 0l2f2ujcqf40w5q8qpd25lyvnrfp049 2448623 2448622 2026-08-17T20:29:18Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-semantic-search-test: apply 2448623 wikitext text/x-wiki av3tyehbb7jgtkxhcgoqw4yr3sjclgd 2448624 2448623 2026-08-17T20:29:26Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-test: apply 2448624 wikitext text/x-wiki hlqexkp8y8umw95f4eb0zb5s949bzpd 2448625 2448624 2026-08-17T20:29:30Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-test: apply 2448625 wikitext text/x-wiki j9pnwf68f7jt1jb2cnddfx028r1645z 2448626 2448625 2026-08-17T20:29:36Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-toolhub: apply 2448626 wikitext text/x-wiki 4tgq55fjdyasrj7h42kyoa9sbkkssvc 2448627 2448626 2026-08-17T20:29:40Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-toolhub: apply 2448627 wikitext text/x-wiki c4kj27f7c0gg815gcvmdx40nviwpq38 2448628 2448627 2026-08-17T20:29:44Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-toolhub: apply 2448628 wikitext text/x-wiki it4ykvsuux7hsoel47xzk3pd3d6uyxz 2448629 2448628 2026-08-17T20:29:51Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-toolhub: apply 2448629 wikitext text/x-wiki 2t6h5i8nfabdya2zfzkujyn6qfc4hit 2448630 2448629 2026-08-17T20:30:01Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-toolhub-test: apply 2448630 wikitext text/x-wiki ghaqj08e0l2xlma2r5pq1syxnzu2m09 2448631 2448630 2026-08-17T20:30:04Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-toolhub-test: apply 2448631 wikitext text/x-wiki 1gwrwn17slg2sgpfck0ntc91t1k00cf 2448632 2448631 2026-08-17T20:30:09Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-toolhub-test: apply 2448632 wikitext text/x-wiki oqdl52a2oes5gwzqig7g5pnn6lyy9xp 2448633 2448632 2026-08-17T20:30:16Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-toolhub-test: apply 2448633 wikitext text/x-wiki kwv9ax0813kniyoix6ey9dnutvffwsn 2448634 2448633 2026-08-17T20:30:20Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-ttmserver: apply 2448634 wikitext text/x-wiki gnykhgre2kkw8c1hkzymts4fnzpvqbq 2448635 2448634 2026-08-17T20:30:24Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-ttmserver: apply 2448635 wikitext text/x-wiki 1syi30zfv437jwoqmj56bfkp5mdcqx3 2448636 2448635 2026-08-17T20:30:29Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-ttmserver: apply 2448636 wikitext text/x-wiki je26561c7xgo97z013gld2cyf7q60lo 2448637 2448636 2026-08-17T20:30:36Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-ttmserver: apply 2448637 wikitext text/x-wiki r1ksbmtrnoh0vy021yww61en9cps2ec 2448638 2448637 2026-08-17T20:30:42Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-ttmserver-test: apply 2448638 wikitext text/x-wiki t0pa76u7fbmhg23oerlfr5n6s2oihh7 2448640 2448638 2026-08-17T20:30:46Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-ttmserver-test: apply 2448640 wikitext text/x-wiki ias9tvpeyppx97gbs1ivgqu1pikd9gx 2448641 2448640 2026-08-17T20:30:51Z Stashbot 7414 cjming@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326361|Remove escaped paths in app site association file (T432412)]] (duration: 13m 28s) 2448641 wikitext text/x-wiki mw15qos6qosbtvzi0mh33b4zw2w3hcf 2448642 2448641 2026-08-17T20:30:54Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-ttmserver-test: apply 2448642 wikitext text/x-wiki bvn3sq7ku6uz4djgy9ovlsdifo8e97q 2448643 2448642 2026-08-17T20:30:57Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-ttmserver-test: apply 2448643 wikitext text/x-wiki 8bmnjfproyxl8nfvzzg6q3xr3t1jkfo 2448644 2448643 2026-08-17T20:33:28Z Stashbot 7414 cjming@deploy1003: Started scap sync-world: Backport for [[gerrit:1324320|[itwiki/slwiki/tgwiki] Remove temporary Wikipedia 25 logos permanently (already reverted) (T414265 T414320 T415307)]] 2448644 wikitext text/x-wiki 3atgbnqtv6p0ljthp0d3x8eg5cz5zh7 2448645 2448644 2026-08-17T20:35:24Z Stashbot 7414 cjming@deploy1003: cjming, superpes: Backport for [[gerrit:1324320|[itwiki/slwiki/tgwiki] Remove temporary Wikipedia 25 logos permanently (already reverted) (T414265 T414320 T415307)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448645 wikitext text/x-wiki q5lv8kklelprrenpc79wmv5jhk849vk 2448646 2448645 2026-08-17T20:36:08Z Stashbot 7414 cjming@deploy1003: cjming, superpes: Continuing with deployment 2448646 wikitext text/x-wiki 7ktvtkqwlz02webuexr4ybdou7fi2mt 2448647 2448646 2026-08-17T20:39:16Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. 2448647 wikitext text/x-wiki bxmi3w97ub7zvbrs5omdt0wu9t1uz8d 2448648 2448647 2026-08-17T20:40:10Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. 2448648 wikitext text/x-wiki k5oomvfsvsboti5z0ewold0lcjfdyd7 2448649 2448648 2026-08-17T20:40:23Z Stashbot 7414 cjming@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324320|[itwiki/slwiki/tgwiki] Remove temporary Wikipedia 25 logos permanently (already reverted) (T414265 T414320 T415307)]] (duration: 06m 55s) 2448649 wikitext text/x-wiki 0tiivvm55vh8phefsthavgt92h9az3d 2448650 2448649 2026-08-17T20:58:35Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:aqs-eqiad: Set storage compatability to UPGRADING — T433026 - eevans@cumin1003 2448650 wikitext text/x-wiki 14qvnco4y6u0ol5rwolfrorigvn6nbp 2448654 2448650 2026-08-17T21:14:39Z Stashbot 7414 maryum: Deployed security fix for T434967 2448654 wikitext text/x-wiki 0lm1e33ixk3pliec0zbongg9n3ni14m 2448657 2448654 2026-08-17T21:21:39Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply 2448657 wikitext text/x-wiki i5w5kx0gh8ae1fx6r061vv0ocpwchr4 2448658 2448657 2026-08-17T21:21:58Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-ipoid-test: apply 2448658 wikitext text/x-wiki d7o05taked5yftse6a4uruhus53mun2 2448659 2448658 2026-08-17T21:23:37Z Stashbot 7414 maryum: Deployed security fix for T433020 2448659 wikitext text/x-wiki kkdn3swaybo7oy7gesgx4i7hra38mby 2448660 2448659 2026-08-17T21:37:13Z Stashbot 7414 maryum: Undeploy security fix for T433020 2448660 wikitext text/x-wiki e9y37jzvhv7aeel74orr1f55dywp2zn 2448661 2448660 2026-08-17T21:39:56Z Stashbot 7414 swfrench@deploy1003: mwscript-k8s job started: purgeList.php # T432412 2448661 wikitext text/x-wiki onw3uync276nwabniz77q14vfia5zvh 2448662 2448661 2026-08-17T21:46:26Z Stashbot 7414 logmsgbot: jforrester Deployed security patch for T435085 2448662 wikitext text/x-wiki ok81tzdx69uknlzskahzqlk88106bew 2448666 2448662 2026-08-17T22:28:31Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.cassandra.roll-restart for nodes matching A:aqs-codfw: Set storage compatability to UPGRADING — T433026 - eevans@cumin1003 2448666 wikitext text/x-wiki jk04a0qy08i9kw857hsmh32yxrv2cnq 2448667 2448666 2026-08-17T23:05:14Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:aqs-codfw: Set storage compatability to UPGRADING — T433026 - eevans@cumin1003 2448667 wikitext text/x-wiki t43fgo8pidjp0wh4bqpszwyisel1phs 2448669 2448667 2026-08-17T23:30:33Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.cassandra.roll-restart for nodes matching A:aqs-eqiad: Set storage compatability to NONE — T433026 - eevans@cumin1003 2448669 wikitext text/x-wiki cf1ofkud2n9jvhsq2bw1l8xelqdyog0 2448672 2448669 2026-08-18T00:04:27Z Stashbot 7414 musikanimal@deploy1003: Started scap sync-world: Backport for [[gerrit:1326373|PersonalDashboard: add newly renamed *ReviewChangesMlModel setting (T422148)]] 2448672 wikitext text/x-wiki mo08shi26jk348ezhgg5ypfzrik580v 2448673 2448672 2026-08-18T00:06:21Z Stashbot 7414 musikanimal@deploy1003: musikanimal: Backport for [[gerrit:1326373|PersonalDashboard: add newly renamed *ReviewChangesMlModel setting (T422148)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448673 wikitext text/x-wiki g9lvos44gpsq76qpsfhwz2pm3fybr83 2448674 2448673 2026-08-18T00:07:12Z Stashbot 7414 musikanimal@deploy1003: musikanimal: Continuing with deployment 2448674 wikitext text/x-wiki 3u5dvmikfcbvgm4161c7j8h0j4wpav4 2448675 2448674 2026-08-18T00:09:35Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:aqs-eqiad: Set storage compatability to NONE — T433026 - eevans@cumin1003 2448675 wikitext text/x-wiki rne5hpcjqm3l44e3jbl7kgupcxr3im9 2448676 2448675 2026-08-18T00:11:34Z Stashbot 7414 musikanimal@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326373|PersonalDashboard: add newly renamed *ReviewChangesMlModel setting (T422148)]] (duration: 07m 07s) 2448676 wikitext text/x-wiki c2jhjmr8ssuvtq33j4xm1mmnzdg7j75 2448677 2448676 2026-08-18T00:38:14Z Stashbot 7414 eevans@cumin1003: START - Cookbook sre.cassandra.roll-restart for nodes matching A:aqs-codfw: Set storage compatability to NONE — T433026 - eevans@cumin1003 2448677 wikitext text/x-wiki jkwybr21b62cc2625gwm9i6ac1ezmjq 2448680 2448677 2026-08-18T01:15:02Z Stashbot 7414 eevans@cumin1003: END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:aqs-codfw: Set storage compatability to NONE — T433026 - eevans@cumin1003 2448680 wikitext text/x-wiki dyozpdtyz3kc80msxjmmd5if1d2m0d9 2448682 2448680 2026-08-18T02:00:52Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2448682 wikitext text/x-wiki nzmy5mcofewo620bv5iydmv8h5e40fm 2448683 2448682 2026-08-18T02:07:38Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 46s) 2448683 wikitext text/x-wiki alw0obmcz88i8ii785tjn1ne3bf3g2w 2448690 2448683 2026-08-18T02:48:51Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1163.eqiad.wmnet with OS bookworm 2448690 wikitext text/x-wiki f6ws8ebbm1chvx7hdkhxggo5epi2f3v 2448691 2448690 2026-08-18T02:49:13Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1164.eqiad.wmnet with OS bookworm 2448691 wikitext text/x-wiki eq9xeonwk9h5r6032iujxze7ti0k9x4 2448692 2448691 2026-08-18T02:49:34Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1165.eqiad.wmnet with OS bookworm 2448692 wikitext text/x-wiki tjimdwolglqfcnvpwbqrsng0ecokl32 2448693 2448692 2026-08-18T02:49:47Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1196.eqiad.wmnet with OS bookworm 2448693 wikitext text/x-wiki 62v6utz28mbau8tc3ckxdh3p6j647sj 2448694 2448693 2026-08-18T02:50:14Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1197.eqiad.wmnet with OS bookworm 2448694 wikitext text/x-wiki 6xhsxwvlpp7i8s5d5vznnvskr5j0d8b 2448697 2448694 2026-08-18T03:03:23Z Stashbot 7414 mwpresync@deploy1003: Started scap sync-world: testwikis to 1.47.0-wmf.16 refs T430835 2448697 wikitext text/x-wiki j5sp3jq8cl99ooesvvli2n2kzs5yvg3 2448698 2448697 2026-08-18T03:04:19Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1163.eqiad.wmnet with reason: host reimage 2448698 wikitext text/x-wiki ggshfbl2jpve30t6ba7iu9570hdhoeb 2448699 2448698 2026-08-18T03:04:28Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1196.eqiad.wmnet with reason: host reimage 2448699 wikitext text/x-wiki dma02x8hjibgc921hggjqezw5ajj7tu 2448700 2448699 2026-08-18T03:04:35Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1197.eqiad.wmnet with reason: host reimage 2448700 wikitext text/x-wiki 7ue382bv7jgf2o4oeimxstj4se3qkje 2448701 2448700 2026-08-18T03:05:00Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1164.eqiad.wmnet with reason: host reimage 2448701 wikitext text/x-wiki 8wi4x99mx4y2otylejp3pr5nxze0jll 2448702 2448701 2026-08-18T03:05:14Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1165.eqiad.wmnet with reason: host reimage 2448702 wikitext text/x-wiki bywp2tdvmwqxmricvpdxe9ne0yapg99 2448703 2448702 2026-08-18T03:09:01Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1163.eqiad.wmnet with reason: host reimage 2448703 wikitext text/x-wiki ewfc5oglhxhzfcrb8m97tpzsuszrdil 2448704 2448703 2026-08-18T03:12:03Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1164.eqiad.wmnet with reason: host reimage 2448704 wikitext text/x-wiki 3hmjxu5uzweylqy39sxyghaefacvgh3 2448705 2448704 2026-08-18T03:15:12Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1196.eqiad.wmnet with reason: host reimage 2448705 wikitext text/x-wiki hsftgxmiuwy2c97jwtijmzgkjvurfte 2448706 2448705 2026-08-18T03:18:44Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1165.eqiad.wmnet with reason: host reimage 2448706 wikitext text/x-wiki l7t0yi7q5hr8ten1aieb3fpayal568a 2448708 2448706 2026-08-18T03:22:27Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1197.eqiad.wmnet with reason: host reimage 2448708 wikitext text/x-wiki b00qekkf5b0qy3nf83e8tfur4v7cws9 2448709 2448708 2026-08-18T03:30:32Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1163.eqiad.wmnet with OS bookworm 2448709 wikitext text/x-wiki htgwwx71d116wzz9ilopxw60hsi8j3i 2448710 2448709 2026-08-18T03:35:13Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1196.eqiad.wmnet with OS bookworm 2448710 wikitext text/x-wiki 6l17wvl6woa971pq0ull2zrhpkob1kf 2448711 2448710 2026-08-18T03:36:17Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1164.eqiad.wmnet with OS bookworm 2448711 wikitext text/x-wiki n0tmbl6r4dzpmlc4mrts9o180qteyu5 2448713 2448711 2026-08-18T03:38:07Z Stashbot 7414 mwpresync@deploy1003: Finished scap sync-world: testwikis to 1.47.0-wmf.16 refs T430835 (duration: 34m 43s) 2448713 wikitext text/x-wiki btvwmw64cdg6pf834ucg1ymzgsv2bve 2448714 2448713 2026-08-18T03:41:46Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1165.eqiad.wmnet with OS bookworm 2448714 wikitext text/x-wiki mrsexqlo8ebhp1svqwt9k6ue8xerk3l 2448715 2448714 2026-08-18T03:45:31Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1197.eqiad.wmnet with OS bookworm 2448715 wikitext text/x-wiki gc2n9mv65jvwor7juv6sax7dbs91lns 2448720 2448715 2026-08-18T04:02:31Z Stashbot 7414 mwpresync@deploy1003: Pruned MediaWiki: 1.47.0-wmf.13 (duration: 02m 23s) 2448720 wikitext text/x-wiki 3boshbse9koirh6nz5u68bzgiy6h32j 2448722 2448720 2026-08-18T04:14:21Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1144.eqiad.wmnet with OS bookworm 2448722 wikitext text/x-wiki pbt4iwxt5zj7c6lmc2zk5syh46xdv2p 2448724 2448722 2026-08-18T04:29:16Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1144.eqiad.wmnet with reason: host reimage 2448724 wikitext text/x-wiki dxuqcnfdhckxqqek7sj4utjcakg682d 2448725 2448724 2026-08-18T04:33:58Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1144.eqiad.wmnet with reason: host reimage 2448725 wikitext text/x-wiki f4z5cheombrqkjwgh8imovl4i91i4qb 2448727 2448725 2026-08-18T04:57:56Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1144.eqiad.wmnet with OS bookworm 2448727 wikitext text/x-wiki c3t1tchxe7wf3va0cv6x6hvecejzxqv 2448728 2448727 2026-08-18T05:06:08Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1155.eqiad.wmnet with OS bookworm 2448728 wikitext text/x-wiki tvl2mmyla9xhsb16630s14cfunvkruf 2448729 2448728 2026-08-18T05:23:11Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1155.eqiad.wmnet with reason: host reimage 2448729 wikitext text/x-wiki fk5lp3n810z64wdvftr432uayow5gdi 2448730 2448729 2026-08-18T05:28:35Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1155.eqiad.wmnet with reason: host reimage 2448730 wikitext text/x-wiki mjxvjlxg5xyq2jhd98c1alybko4mi17 2448731 2448730 2026-08-18T05:38:05Z Stashbot 7414 arnaudb: updating prometheusBearerToken on gerrit 2448731 wikitext text/x-wiki 29ahjqgu4wlsg9m54ozghaags6n7mec 2448732 2448731 2026-08-18T05:50:45Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1155.eqiad.wmnet with OS bookworm 2448732 wikitext text/x-wiki lra9a0vmoqu4fdok78y8qigahbrt1np 2448734 2448732 2026-08-18T06:00:10Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.gerrit.restart-gerrit Restarting Gerrit on gerrit2003 2448734 wikitext text/x-wiki k06g1683ovzaxivnrt1pxqxs65g7z2p 2448735 2448734 2026-08-18T06:02:14Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.gerrit.restart-gerrit (exit_code=0) Restarting Gerrit on gerrit2003 2448735 wikitext text/x-wiki 78qqp3px0i4xbdmem1qa7jvpubdo6yv 2448736 2448735 2026-08-18T06:04:49Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.gerrit.restart-gerrit Restarting Gerrit on gerrit1003 2448736 wikitext text/x-wiki 2s9hfhd5q19vnrzzvy6zert4nu9v5ow 2448737 2448736 2026-08-18T06:06:10Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.gerrit.restart-gerrit (exit_code=0) Restarting Gerrit on gerrit1003 2448737 wikitext text/x-wiki lptldrbj4jbti5dylkb7zqhpo8s68zg 2448738 2448737 2026-08-18T06:06:13Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.gerrit.restart-gerrit Restarting Gerrit on gerrit2002 2448738 wikitext text/x-wiki ayuwxczbkl500oqiyyyru8oqtcker7u 2448739 2448738 2026-08-18T06:09:01Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.gerrit.restart-gerrit (exit_code=0) Restarting Gerrit on gerrit2002 2448739 wikitext text/x-wiki awc77skntrn4us7n2rzwj0593hpm00b 2448742 2448739 2026-08-18T06:41:31Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1054.eqiad.wmnet with OS trixie 2448742 wikitext text/x-wiki ewtwmnxd1uhvewdv4h4fs6062djbygt 2448743 2448742 2026-08-18T06:43:54Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1198.eqiad.wmnet with OS bookworm 2448743 wikitext text/x-wiki 2f2tvlflk5uxeswghkmdqd1yu6evdsv 2448744 2448743 2026-08-18T06:44:53Z Stashbot 7414 XioNoX: upgrade eqsin gnmic to 0.47.0 2448744 wikitext text/x-wiki dolujf8t59w1095pqfinzk3zgnbf91u 2448745 2448744 2026-08-18T06:44:55Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.reimage for host an-worker1194.eqiad.wmnet with OS bookworm 2448745 wikitext text/x-wiki hjsvxmyhnhatfqf3d5tzxibz7cz57es 2448746 2448745 2026-08-18T06:52:19Z Stashbot 7414 filippo@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudvirt1054.eqiad.wmnet with OS trixie 2448746 wikitext text/x-wiki j1hv4tkoica6zcf186gj4arf96jahan 2448747 2448746 2026-08-18T06:53:12Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1055.eqiad.wmnet with OS trixie 2448747 wikitext text/x-wiki cfjnot2edemzp5j1vk1la5gjuy8apx2 2448748 2448747 2026-08-18T06:58:19Z Stashbot 7414 ryankemper@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1198.eqiad.wmnet with reason: host reimage 2448748 wikitext text/x-wiki pb2udwtlcp2icc62mwzyuts32qbsc41 2448749 2448748 2026-08-18T06:59:33Z Stashbot 7414 filippo@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudvirt1055.eqiad.wmnet with OS trixie 2448749 wikitext text/x-wiki fxlyb1t8j687gal93cm7goz6e4j5u51 2448750 2448749 2026-08-18T07:00:01Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1056.eqiad.wmnet with OS trixie 2448750 wikitext text/x-wiki 58n0d9725p1gdeg2lczxpmwqy4evuhd 2448751 2448750 2026-08-18T07:00:19Z Stashbot 7414 filippo@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1056.eqiad.wmnet with OS trixie 2448751 wikitext text/x-wiki nuic3srpefubg1jgjm4ii18a7n2qwuk 2448752 2448751 2026-08-18T07:01:08Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1056.eqiad.wmnet with OS trixie 2448752 wikitext text/x-wiki jzyo6l8j9c9mz6hnuptxpqbvmnfqe1s 2448753 2448752 2026-08-18T07:01:24Z Stashbot 7414 filippo@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1056.eqiad.wmnet with OS trixie 2448753 wikitext text/x-wiki ntqutw3dci87jh212gw5r4wuafwnmxz 2448754 2448753 2026-08-18T07:02:55Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1198.eqiad.wmnet with reason: host reimage 2448754 wikitext text/x-wiki 9e77js5zyad2ube1sllpjekbmudd3ao 2448755 2448754 2026-08-18T07:05:46Z Stashbot 7414 samwilson@deploy1003: Started scap sync-world: Backport for [[gerrit:1326450|InitialiseSettings.php: Enable Bulk OCR on pawikisource (T434648)]] 2448755 wikitext text/x-wiki 8evjc0bf3jc0wbia1iid12fxal0exm2 2448756 2448755 2026-08-18T07:08:00Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reimage for host cloudvirt1057.eqiad.wmnet with OS trixie 2448756 wikitext text/x-wiki bh0rn4fdy5b9usdtr5y4swyk0j2u4vr 2448757 2448756 2026-08-18T07:10:08Z Stashbot 7414 samwilson@deploy1003: samwilson: Backport for [[gerrit:1326450|InitialiseSettings.php: Enable Bulk OCR on pawikisource (T434648)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448757 wikitext text/x-wiki lw409u6l38sly5du6nrgvi0bso1f5pn 2448758 2448757 2026-08-18T07:11:03Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reimage for host ganeti2046.codfw.wmnet with OS bookworm 2448758 wikitext text/x-wiki s7szobtgvsb34rnv5v42pgc0isj7ocr 2448759 2448758 2026-08-18T07:11:08Z Stashbot 7414 samwilson@deploy1003: samwilson: Continuing with deployment 2448759 wikitext text/x-wiki g6htxv1x3951ihbcy8ngfbjf8kzaowj 2448760 2448759 2026-08-18T07:11:49Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1048.eqiad.wmnet 2448760 wikitext text/x-wiki ov6iltw6drfto1r6mirc88g1ivp1q8f 2448761 2448760 2026-08-18T07:17:02Z Stashbot 7414 kevinbazira@deploy1003: helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'llm' for release 'main' . 2448761 wikitext text/x-wiki 4iv6s6vs4mvrco8nm3luf28g6npwunv 2448762 2448761 2026-08-18T07:18:03Z Stashbot 7414 samwilson@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326450|InitialiseSettings.php: Enable Bulk OCR on pawikisource (T434648)]] (duration: 12m 17s) 2448762 wikitext text/x-wiki 6rysoaofgc7ag5n6v6zpzm9qo7w7wu2 2448763 2448762 2026-08-18T07:23:21Z Stashbot 7414 ryankemper@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1198.eqiad.wmnet with OS bookworm 2448763 wikitext text/x-wiki muof46b7ljy0b7t2gqwglmze8m0hcqr 2448764 2448763 2026-08-18T07:23:50Z Stashbot 7414 samwilson@deploy1003: Started scap sync-world: Backport for [[gerrit:701016|InitialiseSettings and -labs: Remove redundant feature flag $wgWikisourceEnableOcr (T285311)]] 2448764 wikitext text/x-wiki bp06uzmmj4h4xxkujzw3g1u8egj9iqx 2448765 2448764 2026-08-18T07:24:01Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti2046.codfw.wmnet with reason: host reimage 2448765 wikitext text/x-wiki 7kicsn66l8k7cqs77tpz8ce8erhd8fd 2448766 2448765 2026-08-18T07:24:13Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1048.eqiad.wmnet 2448766 wikitext text/x-wiki 0to0cuzdzrdfii0pusscyfkvlrozwm0 2448767 2448766 2026-08-18T07:25:01Z Stashbot 7414 aokoth@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: Security Release - T435157 2448767 wikitext text/x-wiki 8qldhe7nkguvgvug1ycaewb717v4ro3 2448768 2448767 2026-08-18T07:25:57Z Stashbot 7414 samwilson@deploy1003: samwilson: Backport for [[gerrit:701016|InitialiseSettings and -labs: Remove redundant feature flag $wgWikisourceEnableOcr (T285311)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448768 wikitext text/x-wiki pw9g0gyvysfryypx1mu9bctn5ovfqee 2448769 2448768 2026-08-18T07:27:19Z Stashbot 7414 samwilson@deploy1003: samwilson: Continuing with deployment 2448769 wikitext text/x-wiki ek98ax1l37kvybzc4nn0nifjsgv1eki 2448770 2448769 2026-08-18T07:28:33Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti2046.codfw.wmnet with reason: host reimage 2448770 wikitext text/x-wiki 6mogvvezhnmz704n5z6ik50ai1xuaeb 2448771 2448770 2026-08-18T07:29:30Z Stashbot 7414 XioNoX: add gnmic 0.47.0 to bookworm and trixie reprepro 2448771 wikitext text/x-wiki 2sxj31jpz69bnbdqa0pbwdryqesw7m1 2448772 2448771 2026-08-18T07:29:45Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1048.eqiad.wmnet 2448772 wikitext text/x-wiki 2flt0ecwfuy8zxeh93jwde4k0dedoql 2448773 2448772 2026-08-18T07:30:14Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1048.eqiad.wmnet 2448773 wikitext text/x-wiki dbb1pszxqie4yrbb4w4ko2l1phfdd39 2448774 2448773 2026-08-18T07:31:36Z Stashbot 7414 samwilson@deploy1003: Finished scap sync-world: Backport for [[gerrit:701016|InitialiseSettings and -labs: Remove redundant feature flag $wgWikisourceEnableOcr (T285311)]] (duration: 07m 47s) 2448774 wikitext text/x-wiki peptne6e3li8ln57lxbp5pb0q8nyddi 2448775 2448774 2026-08-18T07:34:51Z Stashbot 7414 aokoth@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1003.wikimedia.org with reason: Security Release - T435157 2448775 wikitext text/x-wiki aympongwm368dxrrjfxp2j78kjf6ab2 2448776 2448775 2026-08-18T07:38:14Z Stashbot 7414 krinkle@deploy1003: Started scap sync-world: Backport for [[gerrit:1326439|Fix "mathjax_ignore" handling around forcemathmode attribute (T434686)]] 2448776 wikitext text/x-wiki 5p6klnx6rt1iji9n30tgd8gx25n4nas 2448777 2448776 2026-08-18T07:39:01Z Stashbot 7414 aokoth@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab2002.wikimedia.org with reason: Security Release - T435157 2448777 wikitext text/x-wiki 8ce3nb3t25ju4bic4y1sosnvvqhiryg 2448778 2448777 2026-08-18T07:40:15Z Stashbot 7414 krinkle@deploy1003: krinkle: Backport for [[gerrit:1326439|Fix "mathjax_ignore" handling around forcemathmode attribute (T434686)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448778 wikitext text/x-wiki 9qt9xgz05jqnwbztitff3eydwa7s82a 2448779 2448778 2026-08-18T07:44:36Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti2046.codfw.wmnet with OS bookworm 2448779 wikitext text/x-wiki l11z3gej4uqbd6r7amel25cuat5eyoz 2448780 2448779 2026-08-18T07:47:15Z Stashbot 7414 krinkle@deploy1003: krinkle: Continuing with deployment 2448780 wikitext text/x-wiki 9y6wnchp0zn7n8x5tgg8kbsygd6py0o 2448781 2448780 2026-08-18T07:48:55Z Stashbot 7414 aokoth@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab2002.wikimedia.org with reason: Security Release - T435157 2448781 wikitext text/x-wiki pairmm89b75vcm93ph4h9tj0k9v4oe7 2448782 2448781 2026-08-18T07:51:29Z Stashbot 7414 krinkle@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326439|Fix "mathjax_ignore" handling around forcemathmode attribute (T434686)]] (duration: 13m 15s) 2448782 wikitext text/x-wiki 0gky1jxuj3qikx1rzxefhhrz6pfegdi 2448783 2448782 2026-08-18T07:55:00Z Stashbot 7414 aokoth@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1004.wikimedia.org with reason: Security Release - T435157 2448783 wikitext text/x-wiki 2xcrplo0ql42dkjpn7cfqvhz049yfyx 2448785 2448783 2026-08-18T08:05:35Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1289 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96145 and previous config saved to /var/cache/conftool/dbconfig/20260818-080531-marostegui.json 2448785 wikitext text/x-wiki aw71yf2qnyr0so7e2a31crb15hdhqdv 2448786 2448785 2026-08-18T08:05:57Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1289: Pool back 2448786 wikitext text/x-wiki 5rdfzrm8vwbu41a0rtlkuq2yf5vjn7p 2448787 2448786 2026-08-18T08:06:49Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2035.codfw.wmnet 2448787 wikitext text/x-wiki rniksztynmmceb1ayb4i21tr6i7ix72 2448788 2448787 2026-08-18T08:13:34Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2035.codfw.wmnet 2448788 wikitext text/x-wiki 7uy6gbhj0kjakogye2q2p3bzm1xqq7v 2448790 2448788 2026-08-18T08:18:43Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2035.codfw.wmnet 2448790 wikitext text/x-wiki 70eo8qzdu3pupzoe2z9crdnwfc5gh3r 2448791 2448790 2026-08-18T08:18:54Z Stashbot 7414 filippo@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudvirt1057.eqiad.wmnet with OS trixie 2448791 wikitext text/x-wiki fl2jjhgbjosw0b2y8jcgewuz1yh7khy 2448792 2448791 2026-08-18T08:18:56Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2035.codfw.wmnet 2448792 wikitext text/x-wiki 0ylwcqclu03apvvb91ttlruliwkps54 2448793 2448792 2026-08-18T08:20:50Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2036.codfw.wmnet 2448793 wikitext text/x-wiki 61822b0dnkl7nqhuc4f2vvi1wvfoo5u 2448794 2448793 2026-08-18T08:22:38Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1288 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96148 and previous config saved to /var/cache/conftool/dbconfig/20260818-082234-marostegui.json 2448794 wikitext text/x-wiki 3fwswbuf1hmla3fmi0viyngk2kwc75n 2448795 2448794 2026-08-18T08:23:11Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2036.codfw.wmnet 2448795 wikitext text/x-wiki 49jd3k7k25g7t8q93kzpvj11t5lokhb 2448796 2448795 2026-08-18T08:23:34Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.reimage for host an-worker1181.eqiad.wmnet with OS bookworm 2448796 wikitext text/x-wiki hxb7pn47xuxstkk00wk4cmcnmpx5je4 2448797 2448796 2026-08-18T08:23:45Z Stashbot 7414 aokoth@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1004.wikimedia.org with reason: Security Release - T435157 2448797 wikitext text/x-wiki ernwn2p05ujaqoie7070p3ftbwnbkca 2448798 2448797 2026-08-18T08:25:16Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1288: Pool back 2448798 wikitext text/x-wiki agekm7rnhos5xphp5dej7or3gqynpwn 2448799 2448798 2026-08-18T08:28:31Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2036.codfw.wmnet 2448799 wikitext text/x-wiki mkxng4dpv7uizxz3ump16zpbk0rxdmy 2448800 2448799 2026-08-18T08:28:38Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2036.codfw.wmnet 2448800 wikitext text/x-wiki klvb0emtvlytmsr80v0v36s1cfoqf42 2448801 2448800 2026-08-18T08:29:01Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudbackup1001-dev.eqiad.wmnet 2448801 wikitext text/x-wiki ry2187ha0m3c8t3fbmywk4ngefknrly 2448802 2448801 2026-08-18T08:30:33Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2037.codfw.wmnet 2448802 wikitext text/x-wiki mpxpe8jxxdwawc8lgvkdl8r0gmjd6jy 2448803 2448802 2026-08-18T08:30:52Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudbackup1002-dev.eqiad.wmnet 2448803 wikitext text/x-wiki 9n5b78acmp6pepgfi3nj36m9ox5t45z 2448804 2448803 2026-08-18T08:31:04Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudidp2001-dev.codfw.wmnet 2448804 wikitext text/x-wiki p8pd0yiw4b5aspbxkol6pjuusu36qg2 2448805 2448804 2026-08-18T08:32:52Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudbackup1001-dev.eqiad.wmnet 2448805 wikitext text/x-wiki f5hid2qk2x1wbfetfaxunliq3cs1j4a 2448806 2448805 2026-08-18T08:33:15Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1287 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96150 and previous config saved to /var/cache/conftool/dbconfig/20260818-083311-marostegui.json 2448806 wikitext text/x-wiki hixl3pct6z73iy43j7un20xpldtlp8y 2448807 2448806 2026-08-18T08:33:35Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1287: Pool back 2448807 wikitext text/x-wiki lvjs95dmzh62lawofociy3kffh34gsk 2448808 2448807 2026-08-18T08:34:48Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudbackup1002-dev.eqiad.wmnet 2448808 wikitext text/x-wiki ediztpnwf7gmyhbxfj1j7707nxs2mch 2448809 2448808 2026-08-18T08:35:06Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudidp2001-dev.codfw.wmnet 2448809 wikitext text/x-wiki m3ik0cspdkozcd5famxbk3hqqc59bbr 2448810 2448809 2026-08-18T08:51:03Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1289: Pool back 2448810 wikitext text/x-wiki kwur0s7dodu3rq4k50mcvvjn506jf21 2448811 2448810 2026-08-18T08:52:25Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2037.codfw.wmnet 2448811 wikitext text/x-wiki rhkaibiq3mvpevydqmzj8wwpvfj6qu4 2448812 2448811 2026-08-18T08:56:10Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.mysql.multiinstance_reboot for 10 hosts 2448812 wikitext text/x-wiki puur2er22hmzh7dpj6itkd13wfjotgn 2448813 2448812 2026-08-18T08:57:48Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2037.codfw.wmnet 2448813 wikitext text/x-wiki r1mr5e4b4wmcokljrw1xdjdynsljza3 2448814 2448813 2026-08-18T08:57:56Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host bast5005.wikimedia.org 2448814 wikitext text/x-wiki e49u823wx6efnf1romxbcxnba12k5p6 2448815 2448814 2026-08-18T08:57:58Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2037.codfw.wmnet 2448815 wikitext text/x-wiki ffq837j6zxi73xrtzwvqlmx0pm91h6e 2448816 2448815 2026-08-18T09:03:24Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2038.codfw.wmnet 2448816 wikitext text/x-wiki ks2y5ejqkg994ci582kh99daqtg67m4 2448817 2448816 2026-08-18T09:04:09Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5005.wikimedia.org 2448817 wikitext text/x-wiki ehft072gf12j0jkbtdjxm0e4xf15dfc 2448819 2448817 2026-08-18T09:10:25Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1288: Pool back 2448819 wikitext text/x-wiki b7ioa1sc63oqnowm1jsmbsroogjjmuu 2448820 2448819 2026-08-18T09:11:21Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host bast4006.wikimedia.org 2448820 wikitext text/x-wiki ckqp2wdj3j08px9g5632wxfr845vadt 2448826 2448820 2026-08-18T09:13:16Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db1901.eqiad.wmnet 2448826 wikitext text/x-wiki 918scx6blb1lqtcw0metflhfvnoj98t 2448827 2448826 2026-08-18T09:13:19Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448827 wikitext text/x-wiki 8ts7110itzhs2jc9w7hcmvc9ljg0yhb 2448828 2448827 2026-08-18T09:13:23Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2038.codfw.wmnet 2448828 wikitext text/x-wiki jef4vcp728k3o7mambjhek53tz3oura 2448829 2448828 2026-08-18T09:14:17Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reboot-single for host idp-test1005.wikimedia.org 2448829 wikitext text/x-wiki fvctgdm2l3w9xgvj2n3bogykzfqqj5c 2448830 2448829 2026-08-18T09:15:04Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.dns.netbox (exit_code=97) 2448830 wikitext text/x-wiki opcak421rcwgasyoofz0c28tq3lk4q1 2448831 2448830 2026-08-18T09:15:10Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host db1901.eqiad.wmnet 2448831 wikitext text/x-wiki n1ym76tud33d6y4837b5efjswxgyrwe 2448832 2448831 2026-08-18T09:17:31Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.decommission for hosts db1901.eqiad.wmnet 2448832 wikitext text/x-wiki 0o958iakssceys1ydyjh63ua1r5aro3 2448833 2448832 2026-08-18T09:18:13Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test1005.wikimedia.org 2448833 wikitext text/x-wiki 8wsez7xlljxt7kp0rmyi2ulvlmaq9d4 2448834 2448833 2026-08-18T09:18:25Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reboot-single for host idp1005.wikimedia.org 2448834 wikitext text/x-wiki o1n6yula48pr8yxw8i0fi0skh0ia4x4 2448835 2448834 2026-08-18T09:18:42Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2038.codfw.wmnet 2448835 wikitext text/x-wiki ixexvv4sfrcg9wd412ggtqbfmim8gze 2448836 2448835 2026-08-18T09:18:45Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1287: Pool back 2448836 wikitext text/x-wiki 365bncaal4x9outn7odi5i8oi0a2as6 2448838 2448836 2026-08-18T09:19:23Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2038.codfw.wmnet 2448838 wikitext text/x-wiki cil2qshgnuf21sonip3lvjgp9olb8cs 2448839 2448838 2026-08-18T09:19:45Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2039.codfw.wmnet 2448839 wikitext text/x-wiki g3sg251rpta0gptz8q4d91f6t542151 2448840 2448839 2026-08-18T09:21:41Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448840 wikitext text/x-wiki s632ldnle7r7fcw5n6c7oucp640rzys 2448841 2448840 2026-08-18T09:22:22Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp1005.wikimedia.org 2448841 wikitext text/x-wiki 2qzbte701kdrk6z4e9vji1rlc7al0sc 2448842 2448841 2026-08-18T09:22:44Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reboot-single for host idp-test2005.wikimedia.org 2448842 wikitext text/x-wiki q84u5e29kofwqmep9ai4e2zicct6zmn 2448843 2448842 2026-08-18T09:22:54Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2039.codfw.wmnet 2448843 wikitext text/x-wiki abkkzaq1xgnnbqvv3cphesyqsbd3im8 2448845 2448843 2026-08-18T09:26:41Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test2005.wikimedia.org 2448845 wikitext text/x-wiki l4t3a4vusa6ejaym19tethmacqtg66t 2448846 2448845 2026-08-18T09:27:05Z Stashbot 7414 slyngshede@dns1004: START - running authdns-update 2448846 wikitext text/x-wiki tsio2t3500hdalwkmdvg6kyh4ks2d59 2448847 2448846 2026-08-18T09:28:15Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2039.codfw.wmnet 2448847 wikitext text/x-wiki h0i3gjxeh03zwdjhztbwssrww4xksa8 2448848 2448847 2026-08-18T09:28:21Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2039.codfw.wmnet 2448848 wikitext text/x-wiki gxcdke3rfsv9y240fhg24rzjedu289q 2448849 2448848 2026-08-18T09:29:09Z Stashbot 7414 slyngshede@dns1004: END - running authdns-update 2448849 wikitext text/x-wiki 8giwl9xlyu85ed5vqcc9gzr5pupn9bl 2448850 2448849 2026-08-18T09:29:44Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2040.codfw.wmnet 2448850 wikitext text/x-wiki exp6akmriiqbyyenk5z29mwkgfjr768 2448851 2448850 2026-08-18T09:31:14Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db1901.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - fceratto@cumin1003" 2448851 wikitext text/x-wiki p5xpcyry3rtw9lrgj2qon7qu4skb1xc 2448852 2448851 2026-08-18T09:31:33Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db1901.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - fceratto@cumin1003" 2448852 wikitext text/x-wiki 5jurbq9tf7k1de3fw1u1zu970k579jo 2448853 2448852 2026-08-18T09:31:36Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448853 wikitext text/x-wiki 6lqp2nithu74mmewip8vv07t60c7292 2448854 2448853 2026-08-18T09:31:38Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db1901.eqiad.wmnet 2448854 wikitext text/x-wiki hc1dzqwrpz9xcdqgpk9sd1vxbixr6nv 2448855 2448854 2026-08-18T09:31:46Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db1901.eqiad.wmnet 2448855 wikitext text/x-wiki 7nrq9e5jiw7av7b3z70y1xgqsoqb8pa 2448856 2448855 2026-08-18T09:31:49Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448856 wikitext text/x-wiki 3fasf33lqvi0054ajgggfmy95lysub4 2448857 2448856 2026-08-18T09:36:55Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reboot-single for host idp2005.wikimedia.org 2448857 wikitext text/x-wiki i5hy53oesd62uzyl2iu7lz1bb2q3nxx 2448858 2448857 2026-08-18T09:40:54Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp2005.wikimedia.org 2448858 wikitext text/x-wiki hv2im18tepa1to813ezqj2mahtx6548 2448859 2448858 2026-08-18T09:41:10Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2040.codfw.wmnet 2448859 wikitext text/x-wiki a7ymro15ufzir9om2fux1hxdkdkmlz1 2448860 2448859 2026-08-18T09:42:50Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db1901.eqiad.wmnet - fceratto@cumin1003" 2448860 wikitext text/x-wiki 6lod332pps7cxqaror9ijath5h2x0ak 2448861 2448860 2026-08-18T09:42:55Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db1901.eqiad.wmnet - fceratto@cumin1003" 2448861 wikitext text/x-wiki peie2zjsr3sfetxaqh0e89ffmz7wvnb 2448862 2448861 2026-08-18T09:42:57Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448862 wikitext text/x-wiki 0o07kyds0whrnbmc4snzab3wxn8aues 2448863 2448862 2026-08-18T09:42:59Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.wipe-cache db1901.eqiad.wmnet on all recursors 2448863 wikitext text/x-wiki 68dp7v4pf4di6fyrrzplgm4y6htsbc9 2448864 2448863 2026-08-18T09:43:02Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) db1901.eqiad.wmnet on all recursors 2448864 wikitext text/x-wiki 44n07co325jzrf1o6s7p1hf7otgjfd5 2448866 2448864 2026-08-18T09:46:30Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2040.codfw.wmnet 2448866 wikitext text/x-wiki 6iypzr6ycd5gmjmyomckwvsz3cr6k5c 2448867 2448866 2026-08-18T09:46:36Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2040.codfw.wmnet 2448867 wikitext text/x-wiki dldz1kjw9wc4yu43h1x47l3r8dlpim8 2448868 2448867 2026-08-18T09:48:50Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host people1005.eqiad.wmnet 2448868 wikitext text/x-wiki dhp54nypflfqy3p4x2jkeyqgtm8ge54 2448870 2448868 2026-08-18T09:49:07Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host people2004.codfw.wmnet 2448870 wikitext text/x-wiki mum1ejldqd57mz7gfhbhzh6e9uajqlh 2448871 2448870 2026-08-18T09:49:27Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host releases1003.eqiad.wmnet 2448871 wikitext text/x-wiki 6vqs4qo277m8mbaclm5j89bbwg3p9k1 2448872 2448871 2026-08-18T09:49:41Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host releases2003.codfw.wmnet 2448872 wikitext text/x-wiki hqgy3wfqvtmuauj3vlct780jhs899nc 2448873 2448872 2026-08-18T09:50:47Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host releases2003.codfw.wmnet 2448873 wikitext text/x-wiki i5wcarq2t0vs8trp6blaw3d257jna1t 2448874 2448873 2026-08-18T09:52:05Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy7002.magru.wmnet 2448874 wikitext text/x-wiki n0nneb2kkp6k5z42156je7cro6dx325 2448875 2448874 2026-08-18T09:52:43Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host people1005.eqiad.wmnet 2448875 wikitext text/x-wiki cocwuxtj5azqn5kmt9ybwiabfemy25v 2448876 2448875 2026-08-18T09:53:07Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host people2004.codfw.wmnet 2448876 wikitext text/x-wiki 8zk7cup29xeh0uh0mf0zj9z0e4nye53 2448877 2448876 2026-08-18T09:53:17Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy7001.magru.wmnet 2448877 wikitext text/x-wiki lww0j8yq0njdz8zaq001uuvxv9p8hh9 2448878 2448877 2026-08-18T09:53:25Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host releases1003.eqiad.wmnet 2448878 wikitext text/x-wiki 7jvd5ivy9xycdzvjoqd86xd0o6k42ne 2448879 2448878 2026-08-18T09:53:47Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy6002.drmrs.wmnet 2448879 wikitext text/x-wiki 04e6om2bd5tqt78vfh9mlhxu9ie6t81 2448880 2448879 2026-08-18T09:53:59Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy6001.drmrs.wmnet 2448880 wikitext text/x-wiki 4spi6kn6q2489gdwqbenhnsznbdrlaq 2448881 2448880 2026-08-18T09:54:22Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast4006.wikimedia.org 2448881 wikitext text/x-wiki nrcy5e4kj9475da9r1tkhj0w5dcfspl 2448882 2448881 2026-08-18T09:54:24Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.mysql.multiinstance_reboot (exit_code=0) for 10 hosts 2448882 wikitext text/x-wiki poeuwr44dlxvj320eox3vld4fqya5hn 2448884 2448882 2026-08-18T09:57:48Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy6002.drmrs.wmnet 2448884 wikitext text/x-wiki 2qaus50scvnb14w7gnddaqhoqdbnjw9 2448885 2448884 2026-08-18T09:57:51Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy6001.drmrs.wmnet 2448885 wikitext text/x-wiki 0i0hfkzqtpj7nx1iisiuyiunlpqzqkh 2448886 2448885 2026-08-18T09:57:59Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2031.codfw.wmnet 2448886 wikitext text/x-wiki 9bh4rsqqve7o80bey3hi6kv7cc2lxgu 2448887 2448886 2026-08-18T09:58:13Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy7002.magru.wmnet 2448887 wikitext text/x-wiki obrv8jfni6bswrej6xf8nnvnjn4nr5j 2448888 2448887 2026-08-18T09:59:19Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy5004.eqsin.wmnet 2448888 wikitext text/x-wiki 24xal671l9e4mbcamc1qfpv7bb7bdte 2448889 2448888 2026-08-18T09:59:24Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy7001.magru.wmnet 2448889 wikitext text/x-wiki 0bp0huynujxutu1as64fd0qo7n8dy7b 2448890 2448889 2026-08-18T09:59:28Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy5003.eqsin.wmnet 2448890 wikitext text/x-wiki 7yh3ty3vc8cjijesa7g5ra0ys5akr0k 2448891 2448890 2026-08-18T09:59:35Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy4004.ulsfo.wmnet 2448891 wikitext text/x-wiki 04k552sxn5h88tu7o98r1z3lxsybox1 2448893 2448891 2026-08-18T09:59:45Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy4003.ulsfo.wmnet 2448893 wikitext text/x-wiki julkz0hw3l2gk779vjmmkiw3cbdc4lv 2448894 2448893 2026-08-18T10:01:20Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy5004.eqsin.wmnet 2448894 wikitext text/x-wiki 85c8885o3njsctmyxvs3gjkc7f2r7ik 2448895 2448894 2026-08-18T10:01:23Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.clone of db1169.eqiad.wmnet onto db1283.eqiad.wmnet 2448895 wikitext text/x-wiki j8o0ln256vde4b0wbgm6lohirolphle 2448896 2448895 2026-08-18T10:01:26Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1169: Depool db1169.eqiad.wmnet to then clone it to db1283.eqiad.wmnet - marostegui@cumin1003 2448896 wikitext text/x-wiki c3dcyb303eso3a8ediltq02w013n966 2448897 2448896 2026-08-18T10:01:40Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy3002.esams.wmnet 2448897 wikitext text/x-wiki n9tyyvqceqylkj0k17wc7u390znhj8p 2448898 2448897 2026-08-18T10:01:43Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy4004.ulsfo.wmnet 2448898 wikitext text/x-wiki hq0ht9hqxk8worcvf5nt4gl7ha62sui 2448899 2448898 2026-08-18T10:01:55Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy2002.codfw.wmnet 2448899 wikitext text/x-wiki osshcj23plm6d4d438fjvj8wlggmbh1 2448900 2448899 2026-08-18T10:02:01Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1169: Depool db1169.eqiad.wmnet to then clone it to db1283.eqiad.wmnet - marostegui@cumin1003 2448900 wikitext text/x-wiki hgviohr510ht9xd9fgu93ttvteatt2m 2448901 2448900 2026-08-18T10:02:27Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2031.codfw.wmnet 2448901 wikitext text/x-wiki pfdqp53uytzyj00qnk5uer6reco1bnm 2448902 2448901 2026-08-18T10:03:37Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy5003.eqsin.wmnet 2448902 wikitext text/x-wiki 4wxo8ksuveh3dj89ynp79umue7sqxu8 2448903 2448902 2026-08-18T10:03:45Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy4003.ulsfo.wmnet 2448903 wikitext text/x-wiki 7aoxfg67m2k2tvfk52ils60err7hp8s 2448904 2448903 2026-08-18T10:04:05Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy1002.eqiad.wmnet 2448904 wikitext text/x-wiki jwvtyefuzup57gio58fpvte8njpdtzv 2448905 2448904 2026-08-18T10:05:40Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy3002.esams.wmnet 2448905 wikitext text/x-wiki 1c9eufesigr9gqxbgj70o3y35yc6895 2448906 2448905 2026-08-18T10:05:47Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy2002.codfw.wmnet 2448906 wikitext text/x-wiki 245lp8ydwprlcs1j2bwyfbwbmi1girs 2448908 2448906 2026-08-18T10:07:54Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy1002.eqiad.wmnet 2448908 wikitext text/x-wiki afak5s1eblwe3as078flu7ulx3ak8fi 2448909 2448908 2026-08-18T10:08:13Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy1001.eqiad.wmnet 2448909 wikitext text/x-wiki e0tlfmnpcacayn1pyd8zleic01gqz88 2448910 2448909 2026-08-18T10:08:16Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy3001.esams.wmnet 2448910 wikitext text/x-wiki spfchm3xdo0okma35h2auvdijfguggg 2448911 2448910 2026-08-18T10:08:19Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host tcp-proxy2001.codfw.wmnet 2448911 wikitext text/x-wiki cw1io3ldbb11kn7bb30wcityjpegnne 2448912 2448911 2026-08-18T10:08:31Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host zuul2003.codfw.wmnet 2448912 wikitext text/x-wiki jpmti3xfdcyrf3eujniiiv5l3zi5t6m 2448913 2448912 2026-08-18T10:08:35Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2031.codfw.wmnet 2448913 wikitext text/x-wiki r63zgkhbl0aqn0txdg9n4j43bnpgov0 2448914 2448913 2026-08-18T10:08:47Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2031.codfw.wmnet 2448914 wikitext text/x-wiki l80oyszhhop58io08o3rto2wwzvcdp5 2448916 2448914 2026-08-18T10:12:09Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy1001.eqiad.wmnet 2448916 wikitext text/x-wiki b8dr93qyhbxl1v1esafuy9rzgvj1mqr 2448917 2448916 2026-08-18T10:12:16Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy3001.esams.wmnet 2448917 wikitext text/x-wiki m62s9eo85hcahlgs8nz6vbo3au2zfdo 2448918 2448917 2026-08-18T10:12:19Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host tcp-proxy2001.codfw.wmnet 2448918 wikitext text/x-wiki 0wjxfe7wkndhpgk21jtedv5rw6wvkje 2448919 2448918 2026-08-18T10:12:31Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host zuul2003.codfw.wmnet 2448919 wikitext text/x-wiki hbcvehfr13ow9hi5uj9epteobkf6fn0 2448920 2448919 2026-08-18T10:14:37Z Stashbot 7414 blake@deploy1003: Started scap sync-world: no-build deployment for T417800 2448920 wikitext text/x-wiki qi36t2l71hx4mm79mqvgwlh2j7qr7re 2448921 2448920 2026-08-18T10:15:52Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2032.codfw.wmnet 2448921 wikitext text/x-wiki j643w58abtlzbncjcbzxdlq9zs5hg2a 2448922 2448921 2026-08-18T10:16:39Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host zuul2001.codfw.wmnet 2448922 wikitext text/x-wiki 4ni4qf76fyz3fxq0rs7zxg9ic191gt2 2448923 2448922 2026-08-18T10:16:42Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host zuul2002.codfw.wmnet 2448923 wikitext text/x-wiki lx6wpcpc5z9ldtmy8eglkdimwaqv7sr 2448924 2448923 2026-08-18T10:17:52Z Stashbot 7414 blake@deploy1003: Finished scap sync-world: no-build deployment for T417800 (duration: 04m 40s) 2448924 wikitext text/x-wiki pkr1nuewkhkyneu9p5kdyo44xx473fq 2448925 2448924 2026-08-18T10:20:39Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host zuul2001.codfw.wmnet 2448925 wikitext text/x-wiki 4hbz88tsqrq46t7sox2zlbs4st76bwt 2448926 2448925 2026-08-18T10:20:42Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host zuul2002.codfw.wmnet 2448926 wikitext text/x-wiki 4l3hdxjqec2psy2j77ksgg003xralq6 2448927 2448926 2026-08-18T10:22:50Z Stashbot 7414 moritzm: installing Django security updates 2448927 wikitext text/x-wiki sjxlts4i4jdxexjlkuuzhlxwn69jun9 2448929 2448927 2026-08-18T10:24:35Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1286 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96163 and previous config saved to /var/cache/conftool/dbconfig/20260818-102431-marostegui.json 2448929 wikitext text/x-wiki ju0s44ksphorl8fwkzv0bkeq06qir2n 2448930 2448929 2026-08-18T10:24:44Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1286: Pool back 2448930 wikitext text/x-wiki o1ldrbedxdjlxwdo1jdxts4947gixmb 2448932 2448930 2026-08-18T10:27:25Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2032.codfw.wmnet 2448932 wikitext text/x-wiki g9wk586d675rfndpo1b6g5llba1pv11 2448933 2448932 2026-08-18T10:32:24Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.reboot-nodes rolling reboot on A:wikikube-worker-eqiad 2448933 wikitext text/x-wiki 13ajxq0g3404mkc59yl2g1i4nnl0g4m 2448934 2448933 2026-08-18T10:32:51Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1006-1007,1015-1016,1021,1034-1035,1038-1044].eqiad.wmnet 2448934 wikitext text/x-wiki t8myqa61sik9sv9qtocq3wjzg9xif1z 2448935 2448934 2026-08-18T10:33:32Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2032.codfw.wmnet 2448935 wikitext text/x-wiki q0lycf68l4c80zpmvb7t20i48cwbv0a 2448936 2448935 2026-08-18T10:33:40Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2032.codfw.wmnet 2448936 wikitext text/x-wiki k6fl5o7i8idzc21v5fnxpetodzltgkq 2448937 2448936 2026-08-18T10:34:28Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448937 wikitext text/x-wiki cxss1j1lo2f8l7z00rita11acq9kynp 2448938 2448937 2026-08-18T10:36:59Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2448938 wikitext text/x-wiki edubo690r5irtjoovy4z3m3shhc679t 2448939 2448938 2026-08-18T10:37:02Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.ganeti.makevm (exit_code=93) for new host db1901.eqiad.wmnet 2448939 wikitext text/x-wiki 6vxlzhf71dabtvb1hgcw7alfo8jennb 2448940 2448939 2026-08-18T10:37:35Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2047.codfw.wmnet 2448940 wikitext text/x-wiki 550gyrfwvr08exenufwy2ejkwzffp8g 2448941 2448940 2026-08-18T10:40:59Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1006-1007,1015-1016,1021,1034-1035,1038-1044].eqiad.wmnet 2448941 wikitext text/x-wiki cqfwodro0tn4zalrajfdft48igf2oo4 2448942 2448941 2026-08-18T10:41:09Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2047.codfw.wmnet 2448942 wikitext text/x-wiki 27eyv6x4re8gx5wytoeeje13ubawfwd 2448943 2448942 2026-08-18T10:44:26Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.clone of db1168.eqiad.wmnet onto db1282.eqiad.wmnet 2448943 wikitext text/x-wiki k4xusoe0xs1lpjy23mzj4lv4wctse1g 2448944 2448943 2026-08-18T10:44:30Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1168: Depool db1168.eqiad.wmnet to then clone it to db1282.eqiad.wmnet - marostegui@cumin1003 2448944 wikitext text/x-wiki gt0vrplq7xqciqufi4rtdjcalvblgp9 2448945 2448944 2026-08-18T10:44:47Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1168: Depool db1168.eqiad.wmnet to then clone it to db1282.eqiad.wmnet - marostegui@cumin1003 2448945 wikitext text/x-wiki dvtohnb80p1ttltm75cxvxuuspvg6b7 2448946 2448945 2026-08-18T10:46:12Z Stashbot 7414 blake@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2448946 wikitext text/x-wiki 0b7y7zzicdqij31fhemqnyqwdcse6sq 2448947 2448946 2026-08-18T10:46:27Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db1901.eqiad.wmnet 2448947 wikitext text/x-wiki 5k2ae9a847uaraldotg3e00rk9a4nwz 2448948 2448947 2026-08-18T10:46:29Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2047.codfw.wmnet 2448948 wikitext text/x-wiki kzvcf3jt7uybl217t39na9r4wpls8sp 2448949 2448948 2026-08-18T10:46:32Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448949 wikitext text/x-wiki f95xb736lzp75vu22oa8vbmztacrbr7 2448950 2448949 2026-08-18T10:46:35Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2047.codfw.wmnet 2448950 wikitext text/x-wiki 4d44wo3jed6fdujkal1y4nm6f5d2hlg 2448951 2448950 2026-08-18T10:46:45Z Stashbot 7414 blake@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2448951 wikitext text/x-wiki 127uk8ob898g3f49zcye5kqihz87i3l 2448952 2448951 2026-08-18T10:47:39Z Stashbot 7414 blake@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2448952 wikitext text/x-wiki nimaugjqxkzo42qmi00r277xjrv8gk5 2448953 2448952 2026-08-18T10:48:19Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1006-1007,1015-1016,1021,1034-1035,1038-1044].eqiad.wmnet 2448953 wikitext text/x-wiki 4wt627og4ln40f9xmzsqlyvxf78m4n0 2448954 2448953 2026-08-18T10:48:27Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1006-1007,1015-1016,1021,1034-1035,1038-1044].eqiad.wmnet 2448954 wikitext text/x-wiki 280dz3qen96lzv63j6124hpe7vmg7gk 2448955 2448954 2026-08-18T10:48:56Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1045-1050,1056-1057,1064-1066,1073-1075].eqiad.wmnet 2448955 wikitext text/x-wiki l7n0zso64255q3bxaggpf230nfqxh3k 2448956 2448955 2026-08-18T10:49:43Z Stashbot 7414 blake@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2448956 wikitext text/x-wiki 9priqfg4me3pafzq01dx80a6c0r72j8 2448957 2448956 2026-08-18T10:50:05Z Stashbot 7414 blake@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2448957 wikitext text/x-wiki qnc609nb8i8q8ych6qu8crlcby2sckx 2448958 2448957 2026-08-18T10:50:31Z Stashbot 7414 blake@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2448958 wikitext text/x-wiki r63fppcy10tqz2e30n36nb3tc2fim82 2448959 2448958 2026-08-18T10:50:39Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2049.codfw.wmnet 2448959 wikitext text/x-wiki b00sq90ceolri4pcik0hausuf5pe49c 2448960 2448959 2026-08-18T10:51:12Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db1901.eqiad.wmnet - fceratto@cumin1003" 2448960 wikitext text/x-wiki 8jk42o5xmz6xh39092tgljewtjpslwo 2448961 2448960 2026-08-18T10:51:15Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db1901.eqiad.wmnet - fceratto@cumin1003" 2448961 wikitext text/x-wiki a6lu4yfc6vh9wdp86thssl7xlh2ou1v 2448962 2448961 2026-08-18T10:51:17Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448962 wikitext text/x-wiki 5zo25wv0weoo5phank0xweupadh96lm 2448963 2448962 2026-08-18T10:51:20Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.wipe-cache db1901.eqiad.wmnet on all recursors 2448963 wikitext text/x-wiki ikqdg6jav6lmqdmn7lubeiz60c9zovl 2448964 2448963 2026-08-18T10:51:24Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) db1901.eqiad.wmnet on all recursors 2448964 wikitext text/x-wiki frwhl561oqvqnn0ext5i4hg9s2igf7o 2448965 2448964 2026-08-18T10:53:35Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host failoid2003.codfw.wmnet 2448965 wikitext text/x-wiki 8qid7gd8xplaqrpa55jufwx8vl6jeog 2448967 2448965 2026-08-18T10:54:45Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1181.eqiad.wmnet with reason: host reimage 2448967 wikitext text/x-wiki a9uqlz7qza618qp7oieayoxln6pbz82 2448968 2448967 2026-08-18T10:57:24Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host failoid2003.codfw.wmnet 2448968 wikitext text/x-wiki m8f4fwizaq7rwyiw9u157jlbdmdptl6 2448969 2448968 2026-08-18T10:59:02Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host failoid1003.eqiad.wmnet 2448969 wikitext text/x-wiki 2kfeufqitvx6mg33nowrxjgdc6odd2v 2448970 2448969 2026-08-18T10:59:08Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2049.codfw.wmnet 2448970 wikitext text/x-wiki bmtmfisz1pouu3imqelc761pvwvvllf 2448971 2448970 2026-08-18T10:59:20Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1181.eqiad.wmnet with reason: host reimage 2448971 wikitext text/x-wiki 5aujdy6fecyautte7lpb9ja1yi6830s 2448972 2448971 2026-08-18T11:01:37Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1045-1050,1056-1057,1064-1066,1073-1075].eqiad.wmnet 2448972 wikitext text/x-wiki tnl2vyrfi8wouchybqdnyg16q9lt00y 2448973 2448972 2026-08-18T11:02:55Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host failoid1003.eqiad.wmnet 2448973 wikitext text/x-wiki emqbz9n5d9itczbquw8afnn7r3k02oc 2448974 2448973 2026-08-18T11:04:40Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2049.codfw.wmnet 2448974 wikitext text/x-wiki 9i5n37wgwar8li9lp0ffnzar5wpnkv3 2448975 2448974 2026-08-18T11:05:28Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2049.codfw.wmnet 2448975 wikitext text/x-wiki 8o4zk2hjn3tiiaw67gitm2ptimat1wv 2448976 2448975 2026-08-18T11:06:25Z Stashbot 7414 btullis@cumin1003: START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1194.eqiad.wmnet with reason: host reimage 2448976 wikitext text/x-wiki 0qck3m41jl9bbz13d48g489hp5dm5y4 2448977 2448976 2026-08-18T11:06:29Z Stashbot 7414 btullis@cumin1003: END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on an-worker1194.eqiad.wmnet with reason: host reimage 2448977 wikitext text/x-wiki o6zgja92hmmx0rkhtd0ut5jv71bryd9 2448978 2448977 2026-08-18T11:07:17Z Stashbot 7414 moritzm: installing PHP 8.4 security updates 2448978 wikitext text/x-wiki k00kuwid8wscb56u4q39f6se0z8v4by 2448979 2448978 2026-08-18T11:08:23Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host build2004.codfw.wmnet 2448979 wikitext text/x-wiki 6dx1g21sslsduqbtcmxteb1x35akh1v 2448980 2448979 2026-08-18T11:09:18Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1045-1050,1056-1057,1064-1066,1073-1075].eqiad.wmnet 2448980 wikitext text/x-wiki agript7q93924naoaolxuj9wujf7o1r 2448981 2448980 2026-08-18T11:09:27Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1045-1050,1056-1057,1064-1066,1073-1075].eqiad.wmnet 2448981 wikitext text/x-wiki l77wux9y626bpc7eqaq6xy9q45njnls 2448982 2448981 2026-08-18T11:09:53Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1286: Pool back 2448982 wikitext text/x-wiki pxx4x70ek83xakid1f8uyas1b0f4kce 2448983 2448982 2026-08-18T11:09:58Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1076-1081,1084-1087,1093-1095,1113].eqiad.wmnet 2448983 wikitext text/x-wiki ikqe2g8rmoqx8d7g67374hzg2wu0ddy 2448984 2448983 2026-08-18T11:10:36Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2050.codfw.wmnet 2448984 wikitext text/x-wiki s5m1xcrad85yg2eytiv28gx3f7j5kla 2448985 2448984 2026-08-18T11:14:26Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host build2004.codfw.wmnet 2448985 wikitext text/x-wiki 3x7tvahxnb8gay9ix5p7wqqhwo9jlds 2448986 2448985 2026-08-18T11:15:53Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2050.codfw.wmnet 2448986 wikitext text/x-wiki p45lotxpgnhk6qdfklf7in5esltl2fi 2448987 2448986 2026-08-18T11:18:13Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1076-1081,1084-1087,1093-1095,1113].eqiad.wmnet 2448987 wikitext text/x-wiki 9l12jcc3yg99586jonzzr9p93yurugd 2448988 2448987 2026-08-18T11:21:14Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2050.codfw.wmnet 2448988 wikitext text/x-wiki f34u2ywh57olmronodwlrvm40lxodxi 2448989 2448988 2026-08-18T11:21:21Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2050.codfw.wmnet 2448989 wikitext text/x-wiki eg8cq05txy5tsoeuuytc4f8xya36vpm 2448990 2448989 2026-08-18T11:23:56Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1181.eqiad.wmnet with OS bookworm 2448990 wikitext text/x-wiki 7jw3hzjbff5ls95h906fj94kgnbymro 2448991 2448990 2026-08-18T11:24:58Z Stashbot 7414 btullis@cumin1003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1194.eqiad.wmnet with OS bookworm 2448991 wikitext text/x-wiki 9j7bsjir7w5po2h6efiwt2ru9rrosyj 2448992 2448991 2026-08-18T11:25:16Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1168: Pool db1168.eqiad.wmnet in after cloning 2448992 wikitext text/x-wiki bo9tizalcxr21hczgp3tva2g1pkm83f 2448993 2448992 2026-08-18T11:26:51Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1076-1081,1084-1087,1093-1095,1113].eqiad.wmnet 2448993 wikitext text/x-wiki 7x4x9zu3x9vo0mi07re35enzlkksha4 2448994 2448993 2026-08-18T11:27:00Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1076-1081,1084-1087,1093-1095,1113].eqiad.wmnet 2448994 wikitext text/x-wiki 7od30a73iy33dp52vyim1yva19175zk 2448995 2448994 2026-08-18T11:27:29Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1114-1127].eqiad.wmnet 2448995 wikitext text/x-wiki pzavosuh5ptssrei4eghedm0h35vdi3 2448996 2448995 2026-08-18T11:30:55Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.kafka.roll-restart-reboot-brokers rolling reboot on A:kafka-test-eqiad 2448996 wikitext text/x-wiki ra80sfkxh1o1bb1fi7cus0asg2dbub4 2448998 2448996 2026-08-18T11:35:03Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448998 wikitext text/x-wiki a0idx1naqdpixdj8qwfharzjrb2jb9b 2448999 2448998 2026-08-18T11:35:50Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1169: Pool db1169.eqiad.wmnet in after cloning 2448999 wikitext text/x-wiki heq7vw5a6sstjqdoyk1bihm2d2bhcdw 2449000 2448999 2026-08-18T11:35:59Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.ganeti.makevm (exit_code=93) for new host db1901.eqiad.wmnet 2449000 wikitext text/x-wiki i14ddzelgxmb1j6s71vlxzab86f0h44 2449001 2449000 2026-08-18T11:36:07Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1114-1127].eqiad.wmnet 2449001 wikitext text/x-wiki 3m26gavs64ehy81vwx0zwwdukrr0zup 2449002 2449001 2026-08-18T11:36:45Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2025.codfw.wmnet 2449002 wikitext text/x-wiki 9em6owsd3zk1hv42fid8k3alp0ki2nb 2449003 2449002 2026-08-18T11:39:57Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db2901.codfw.wmnet 2449003 wikitext text/x-wiki dvm0g0s56slgh43oo77si7jwc4wqtn2 2449004 2449003 2026-08-18T11:40:00Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host db2901.codfw.wmnet 2449004 wikitext text/x-wiki lgd3lua7sdvfbq4xjqwgbt4y8e90whd 2449006 2449004 2026-08-18T11:45:34Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1114-1127].eqiad.wmnet 2449006 wikitext text/x-wiki p281b49mthnhdzmfmsrbqo8f212rkut 2449007 2449006 2026-08-18T11:45:42Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1114-1127].eqiad.wmnet 2449007 wikitext text/x-wiki 5nerv52gr8hfejqe5fmtvtnzjtfqbmp 2449008 2449007 2026-08-18T11:46:11Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1128-1134,1142-1148].eqiad.wmnet 2449008 wikitext text/x-wiki gosw03h1wn7sfz6b28uah9jsz5qqemu 2449009 2449008 2026-08-18T11:50:52Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2025.codfw.wmnet 2449009 wikitext text/x-wiki rrgo6zu3ns6eh2w1m4fpl4ot9v4lz1q 2449010 2449009 2026-08-18T11:54:15Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1128-1134,1142-1148].eqiad.wmnet 2449010 wikitext text/x-wiki 6iyg9sn7lzzwy2kxlw22n55cq9mqbl7 2449011 2449010 2026-08-18T11:54:51Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti2046.codfw.wmnet 2449011 wikitext text/x-wiki 5l8dnng2sja3b22a4hkv7w55m1n7dgh 2449013 2449011 2026-08-18T11:57:04Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2025.codfw.wmnet 2449013 wikitext text/x-wiki axh6fx8ob4pgpf95gimyemtie9oy2uk 2449014 2449013 2026-08-18T11:57:10Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2025.codfw.wmnet 2449014 wikitext text/x-wiki rsc5p61ni7l4116ob0cxxtpbh9t1f57 2 8581 2448684 2443350 2026-08-18T02:18:23Z Quiddity 1884 remove merged codesearch-beta 2448684 wikitext text/x-wiki j86wyju9xcawcr6yqo0j9vq8djhf96s 0 17290 2448444 2447751 2026-08-17T16:30:50Z Stashbot 7414 James_F: Zuul: [mediawiki/extensions/WP25EasterEggs] Archive repository, for T418134 2448444 wikitext text/x-wiki qjk6maiwci3o9bofzqkev9d3gu7rmy7 2448529 2448444 2026-08-17T18:08:18Z Stashbot 7414 brennen: Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/113 2448529 wikitext text/x-wiki chvt7am0uqe7q8x8ab04tdlanufhcmr 2448651 2448529 2026-08-17T21:01:20Z Stashbot 7414 brennen: Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/118 (T413817, T407430) 2448651 wikitext text/x-wiki kn0a6440z830yye3g4l3pti0b5drwn4 2448653 2448651 2026-08-17T21:09:44Z Stashbot 7414 brennen: Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/115 2448653 wikitext text/x-wiki jlvd1zhperl6o5l25kije59fnw4y3d1 2448844 2448653 2026-08-18T09:24:18Z Stashbot 7414 hashar: zuul: restarted zuul-web 2448844 wikitext text/x-wiki riapi0ii8rlto0ec75r5tuo8mddn1rx 0 22777 2448211 2447268 2026-08-17T13:31:37Z FCeratto-WMF 39104 Add tip 2448211 wikitext text/x-wiki 822zoy35y3drzwtxibq272l7tl0r3x7 0 26586 2448670 2325304 2026-08-17T23:33:17Z ADenisse-WMF 31462 /* Generating a key for a new identity or update an existing one */ 2448670 wikitext text/x-wiki 24iro792rcwtw3m2wox4drs9uc44oc0 498 30942 2448652 2447765 2026-08-17T21:07:10Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.roll_reboot_mons (T434750) 2448652 wikitext text/x-wiki cwr6josigbm8rdup2o7gqc9y95t8xgc 2448655 2448652 2026-08-17T21:19:52Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.roll_reboot_mons (exit_code=0) (T434750) 2448655 wikitext text/x-wiki re2yotqxtauvoql7ltmfiwq37x62why 2448656 2448655 2026-08-17T21:20:26Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.ceph.roll_reboot_osds (T434750) 2448656 wikitext text/x-wiki 80ooi392im4d9tpysp6u0hfxx8s2b9d 2448679 2448656 2026-08-18T01:13:57Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.ceph.roll_reboot_osds (exit_code=0) (T434750) 2448679 wikitext text/x-wiki 7ne7j4d8q812pe816jgno61dswq0ywo 0 90760 2449012 2446445 2026-08-18T11:56:02Z MLechvien-WMF 49197 Fix broken links 2449012 wikitext text/x-wiki 739wi56m6gpdhamxwbrva3e4lijl43n 0 442288 2448685 2246191 2026-08-18T02:19:47Z Krinkle 44 /* Caution */ 2448685 wikitext text/x-wiki aqsxpfrcp36tp29l07ln7lhfq09a5e9 2448686 2448685 2026-08-18T02:20:18Z Krinkle 44 /* Fixme */ 2448686 wikitext text/x-wiki iv5ydgsgtwm4ask8yggz76i1nhryki2 2448687 2448686 2026-08-18T02:20:25Z Krinkle 44 /* Fixme */ 2448687 wikitext text/x-wiki quv8bfrmkw92o7kmwqcij1919n9i7uq 2448688 2448687 2026-08-18T02:20:45Z Krinkle 44 Reverted edits by [[Special:Contributions/Krinkle|Krinkle]] ([[User talk:Krinkle|talk]]) to last revision by [[User:FNegri-WMF|FNegri-WMF]] 2246191 wikitext text/x-wiki tb8n23fxn43p61v0mhtc6zn7g7l92v6 498 443800 2448522 2418772 2026-08-17T17:45:02Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed fe67524e52 (migrate to Toolforge Components Service for webservice and backround runner) 2448522 wikitext text/x-wiki d18ndujv9uyjdpnsunfcuprcrgue58k 498 443946 2448488 2438289 2026-08-17T17:08:07Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed b6485e5984 (l10n updates: el, fa, fi, ga, gl, ja, lb, ms, nb, sl, tr – a few had queued up due to T432838) [actually deployed bb18af2eb1, temporary T431146 branch rebased on top of that] 2448488 wikitext text/x-wiki 2j68pb0se9xosnxw97me8uubiuqr44f 498 443949 2448568 2447806 2026-08-17T19:34:02Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 12202cd (Refactor: columns own their SELECT variables for drill-down queries) 2448568 wikitext text/x-wiki ckkm6hnt5871egnfpz7i6d36paifipq 2448569 2448568 2026-08-17T19:34:03Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 3f69cf4 (Extract _format_value_sparql helper for value-to-SPARQL-term conversion) 2448569 wikitext text/x-wiki ews64i6mw53q05soqo5e93cdnfvckyj 2448570 2448569 2026-08-17T19:34:05Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy f9043eb (DescriptionColumn: show the description value in positive drill-down) 2448570 wikitext text/x-wiki nqcbk0nwq8e5ytyzedolgpbky3ghy8q 2448571 2448570 2026-08-17T19:34:09Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 3976414 (ReferenceColumn: show statement value in negative drill-down) for T428637 2448571 wikitext text/x-wiki gb4lapbzeszy21xk1dxaionpxgw4uag 2448572 2448571 2026-08-17T19:34:10Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 589a9a3 (ReferenceColumn: show reference value in positive drill-down) for T428636 2448572 wikitext text/x-wiki t31p45t176i78r8p3tfp4qtb85789vp 2448598 2448572 2026-08-17T20:22:19Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 853a540 (Add --page argument to CLI for single-page updates) 2448598 wikitext text/x-wiki edqhhjua9tkmri6zj1s31wt2zjx3g2e 2448599 2448598 2026-08-17T20:22:20Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy bbf2c99 (Set name in toolinfo record to toolforge-$TOOL_NAME to avoid duplicates) 2448599 wikitext text/x-wiki amlielkw1qhk6ral4377l89djklxu1g 2448600 2448599 2026-08-17T20:22:22Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 1b3b5ec (Update toolinfo.json record, based on latest 1.2.2 schema) 2448600 wikitext text/x-wiki sj99zhun7iaw7q0ebxvj0rpbw4815w5 2448821 2448600 2026-08-18T09:12:58Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 66c6495 (Fix tool type in toolinfo.json, as lists are not accepted) 2448821 wikitext text/x-wiki hjwnkn3ba0ypbj604i64i5nrhfo899o 2448822 2448821 2026-08-18T09:12:59Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 5da6acb (Add pre-commit hook to validate toolinfo.json against Toolhub schema) 2448822 wikitext text/x-wiki lnfl8h6bhc5670hwt0tsnpff8kha8dk 2448823 2448822 2026-08-18T09:13:01Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 5cc67af (Add pre-commit hook to validate docker-compose.yml against Compose schema) 2448823 wikitext text/x-wiki 5gs60uh8oi2gezst5rkjzxjsj99l0x3 2448824 2448823 2026-08-18T09:13:03Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy fd89068 (Reformat pyproject.toml using pyproject-fmt) 2448824 wikitext text/x-wiki hs778mre64v393cym48b93xf0l1w9z4 2448825 2448824 2026-08-18T09:13:05Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy b75abe2 (Add pre-commit hook to reformat pyproject.toml using pyproject-fmt) 2448825 wikitext text/x-wiki 2ja7zbl37uajuz65pn16l08t7yeyw5n 0 444677 2448466 2436500 2026-08-17T16:49:30Z DLynch (WMF) 6438 Add editcheck-headless as 4693 2448466 wikitext text/x-wiki itlcx017oisxq048lyhm7r28tiak1mb 0 445864 2449005 2443398 2026-08-18T11:44:18Z DCaro (WMF) 18271 /* Merge the MR and create the release */ updated release process 2449005 wikitext text/x-wiki 6w242hm1eoeltrn0s4ulrc3g2tjcl4h 498 446686 2448915 2440291 2026-08-18T10:11:42Z Stashbot 7414 taavi: remove hold on grafana package 2448915 wikitext text/x-wiki 0bgkk5ba061lvx25vrj724k39ge1d0z 0 449160 2448671 2447807 2026-08-18T00:00:40Z Dexbot 30554 Bot: Updating the report 2448671 wikitext text/x-wiki 4tblndgqkjg2nd68nvpmchpxizbc6lb 0 452340 2448668 2420196 2026-08-17T23:09:19Z Aaron Schulz (WMF) 47201 update link 2448668 wikitext text/x-wiki 37z25y4i2md6w0hyxjltz8z9p7b8ygr 0 453597 2448928 2447076 2026-08-18T10:24:25Z DCaro (WMF) 18271 /* 2026 */ 2448928 wikitext text/x-wiki 7t39z7n451bwylh1ea8xzs7m6t3etq2 116 453906 2448178 2447783 2026-08-17T12:48:30Z Gitlabaccountapprovalbot 37332 @lfs was approved. 2448178 wikitext text/x-wiki apxb79c4p7kdgj7rfqj1gdi4um9yq39 2448535 2448178 2026-08-17T18:33:11Z Gitlabaccountapprovalbot 37332 ashishkaranam was rejected. 2448535 wikitext text/x-wiki n4gdwcs2r4s529pak3ge9srfomhha8i 2448537 2448535 2026-08-17T18:39:27Z Gitlabaccountapprovalbot 37332 @erlanger16e was approved. 2448537 wikitext text/x-wiki 6pxrp5c3ctc3k2eez0yjvgqt9fcoaqq 2448997 2448537 2026-08-18T11:33:19Z Gitlabaccountapprovalbot 37332 wladek92 was rejected. 2448997 wikitext text/x-wiki 2feki6vn78c32j3hmwpj9p0si9qeu5s 12 458860 2448931 2420940 2026-08-18T10:25:03Z DCaro (WMF) 18271 /* Current supported and non-supported features */ 2448931 wikitext text/x-wiki h8ylw46k538f6pp3o4gl9vyielfcnu6 0 459039 2448516 2360799 2026-08-17T17:36:38Z AStein-WMF 39485 add pipelines section 2448516 wikitext text/x-wiki m5fmsii0lhhftxuumrj9l34hunt1v69 0 459699 2448260 2447812 2026-08-17T14:08:27Z Stashbot 7414 damilare: civicrm upgraded from ced9a157 to 94367c2d 2448260 wikitext text/x-wiki slvurkx0vio6qg3go3m7l1ny2ssrp5s 2448566 2448260 2026-08-17T19:31:26Z Stashbot 7414 larssandergreen: civicrm upgraded from 94367c2d to 1ce07d30 2448566 wikitext text/x-wiki 3eq0qp2ru9wrdyppixwsfumdwqdje35 2448639 2448566 2026-08-17T20:30:44Z Stashbot 7414 ejegg: standalone (IPN listener) SmashPig upgraded from 3acbe469 to 49c58f98 2448639 wikitext text/x-wiki tdtlpi9bu5zg9l7is6cvymjvlz1uabf 2448663 2448639 2026-08-17T22:06:42Z Stashbot 7414 ejegg: payments-wiki upgraded from a06ae998 to bfc18959 2448663 wikitext text/x-wiki c2sjc6abhvtjnpfzn0q7ymdrfao9emo 2448678 2448663 2026-08-18T01:08:17Z Stashbot 7414 eileen: SmashPig upgraded from 49c58f98 to 9930737e 2448678 wikitext text/x-wiki 74p19ccw03kytm2rht09l2hjlq3kmef 2448681 2448678 2026-08-18T01:17:54Z Stashbot 7414 eileen: config revision changed from 902dba91 to d8e0fd79 2448681 wikitext text/x-wiki ky1r3z5y46v31bmz3hlnv1wu5b476jz 2448707 2448681 2026-08-18T03:20:21Z Stashbot 7414 eileen: civicrm upgraded from 1ce07d30 to 49047af1 2448707 wikitext text/x-wiki hdi14tznmhukxtlqht2h5w1cbgifsl1 2448721 2448707 2026-08-18T04:09:13Z Stashbot 7414 eileen: SmashPig upgraded from 9930737e to 89909858 2448721 wikitext text/x-wiki qxiqaxmyvaxxwz8djeo8opezv3xf9nq 0 459895 2448966 2426112 2026-08-18T10:53:45Z BJensen-WMF 48942 /* Deployment to staging */ Correct test run location. 2448966 wikitext text/x-wiki i8hzpumnf3jqi1femd3an092v24p557 0 460052 2448291 2432401 2026-08-17T14:24:50Z MPopov (WMF) 5891 Notes on troubleshooting access 2448291 wikitext text/x-wiki atqqqncksmo3wdcfrm3on9mgjagpix5 0 460234 2448482 2439429 2026-08-17T16:59:27Z JVanderhoop-WMF 41069 Removed traffic limit exception references 2448482 wikitext text/x-wiki ntjo3fhx4eu7k2fsvzrrtjx3h26osud 0 460375 2448573 2447803 2026-08-17T19:35:18Z AWesterinen-WMF 54189 /* wikibase:box */ 2448573 wikitext text/x-wiki 9xi04zppskl5no5ap54gxmyd3pq8rj5 2448575 2448573 2026-08-17T19:36:21Z AWesterinen-WMF 54189 /* wikibase:box */ 2448575 wikitext text/x-wiki 0ul0ovsujn89fbaah4cch3ukg0tfxq6 2448589 2448575 2026-08-17T20:03:41Z AWesterinen-WMF 54189 /* Example 2 */ 2448589 wikitext text/x-wiki fnmcbdhvjhhirioi4ui8wcx4yi746t8 2448689 2448589 2026-08-18T02:22:50Z Quiddity 1884 <syntaxhighlight lang="text"> to match prior 3 examples (perhaps this and the others should be lang="sparql" instead?) 2448689 wikitext text/x-wiki o9e4g3ipo4948oocqo1b2vwna27j4rd 0 460376 2448453 2447761 2026-08-17T16:38:08Z AWesterinen-WMF 54189 /* bd:sample */ 2448453 wikitext text/x-wiki tlesd9fn3q6tj1ovvvtebs109ckm2es 2448712 2448453 2026-08-18T03:37:38Z AWesterinen-WMF 54189 /* Query Rewrite Pattern for wikibase:decodeURI */ 2448712 wikitext text/x-wiki 8vfcjk6vfkfu0419ur5app3vk75f4yv 2448717 2448712 2026-08-18T03:47:57Z AWesterinen-WMF 54189 /* wikibase:label */ 2448717 wikitext text/x-wiki tluev080ghfqnsavzqe3gpdr14c2peq 2448718 2448717 2026-08-18T04:01:35Z AWesterinen-WMF 54189 /* Named Subqueries */ 2448718 wikitext text/x-wiki lhqt8a95r6kh2fhnks15il0854ikljd 2448719 2448718 2026-08-18T04:02:00Z AWesterinen-WMF 54189 /* wikibase:isSomeValue() */ 2448719 wikitext text/x-wiki jo1mzslgp1474yskugu4nar2az3chma 2448723 2448719 2026-08-18T04:22:20Z AWesterinen-WMF 54189 /* wikibase:isSomeValue() */ 2448723 wikitext text/x-wiki gxi3vab0xdze16suxhdcz9nhji67s64 2448726 2448723 2026-08-18T04:43:15Z AWesterinen-WMF 54189 /* bd:sample */ 2448726 wikitext text/x-wiki 8kkvm7okiwclbgfcawhytv9qrek7808 0 460515 2448818 2447571 2026-08-18T09:09:59Z GGoncalves-WMF 42848 Reinstate {draft}. 2448818 wikitext text/x-wiki sd4s3rcz7wzkc18rbipx7nf8wlho7bm 0 460538 2448740 2446638 2026-08-18T06:36:01Z Arthur Taylor (WMDE) 37356 /* Import Statements with Violations In progress */ Update now that the patch has been merged 2448740 wikitext text/x-wiki p6xmvws6llihhhh8e0lxsz8khycme8s 2448741 2448740 2026-08-18T06:36:19Z Arthur Taylor (WMDE) 37356 /* Import Statements with Violations In progress */ Remove 'in progress' header 2448741 wikitext text/x-wiki q17efi9xbrfwtab2d18i2ofxwlad2la 2 460540 2448525 2446917 2026-08-17T17:59:00Z Ottomata 88 2448525 wikitext text/x-wiki kmcrfr6hzbxwmd10h91ck6s7lrh62xw 0 460559 2448518 2026-08-17T17:37:18Z AStein-WMF 39485 Created blank page 2448518 wikitext text/x-wiki phoiac9h4m842xq45sp7s6u21eteeq1 2448527 2448518 2026-08-17T18:04:05Z AStein-WMF 39485 2448527 wikitext text/x-wiki 4368bvtrpqld5t5slyr116laqk4ycux 2448531 2448527 2026-08-17T18:16:01Z AStein-WMF 39485 2448531 wikitext text/x-wiki 2w3pocbkftwbtq9sh7dbwslvs2c7rtd 2448540 2448531 2026-08-17T18:47:34Z AStein-WMF 39485 2448540 wikitext text/x-wiki gf77nkvp5wexbcqrmmikv5ccgz6wzya 2448541 2448540 2026-08-17T18:48:22Z AStein-WMF 39485 2448541 wikitext text/x-wiki f9j6j6stgbmbuyao2x1yrk10t0ebitt 2448545 2448541 2026-08-17T18:54:18Z AStein-WMF 39485 2448545 wikitext text/x-wiki llc0gngfhwq14pjkgd17amy5gsyquav 0 460560 2448664 2026-08-17T22:08:00Z BKing (WMF) 29017 Created page with "=Opensearch 3.7 K8s: special indices blocking shard recovery= I spotted some odd behavior during an upgrade in [https://phabricator.wikimedia.org/T435125 T435125] . Noting it here just in case we see it again. <code>opensearch-semantic-search</code> was in yellow status. I noticed a few unassigned shards: <syntaxhighlight lang="bash">[bking@deploy1003 codfw] ~$ curl https://opensearch-semantic-search.svc.codfw.wmnet:30443/_cat/shards | grep -vi started % Total %..." 2448664 wikitext text/x-wiki o2tau3z6c8qmrh66zfb9lhe526c4n6g 2448665 2448664 2026-08-17T22:11:21Z BKing (WMF) 29017 /* Opensearch 3.7 K8s: special indices blocking shard recovery */ 2448665 wikitext text/x-wiki 98nhvpwaeh8lehu292k9r6rrxdl2d4x 0 460561 2448837 2026-08-18T09:18:54Z KCVelaga (WMF) 38150 Created page with "{{draft}} The following tables contain enriched and pre-aggregated datasets of traffic to [[mw:Wikimedia_APIs|Wikimedia public APIs]]. The data is filtered from webrequest, keeps only API requested, is enriched with additional columns of interest that are regularly used for API-related analyses and metrics. == Datasets == Currently, we have the following tables in this regard: == api_requests_refined == == mrt_api_requests_hourly == == mediawiki_api_request == ==..." 2448837 wikitext text/x-wiki hf367xsjjrbc2u527gf4yaqsyni7i83