Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.46.0-wmf.23 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2400174 2399958 2026-04-09T13:54:12Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2400174 wikitext text/x-wiki jku6kf3zpaptnh518sqq1zaveqlbckx 2400181 2400174 2026-04-09T14:09:13Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2400181 wikitext text/x-wiki 06eec4sj37y06ez8m7su2eh6xeieopm 2400431 2400181 2026-04-10T08:33:25Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component istio-gateway 2400431 wikitext text/x-wiki bsn3gyoyk9yz5idfhx5fvjm3bsb1b3v 2400432 2400431 2026-04-10T08:33:38Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component istio-gateway 2400432 wikitext text/x-wiki 2w0f1ft2b3x7ocastt1tbkgg1enqc1j 2400435 2400432 2026-04-10T08:41:46Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component istio-gateway 2400435 wikitext text/x-wiki gg5hs3b9dxd298n9236ao9ujl552iuv 2400436 2400435 2026-04-10T08:41:58Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component istio-gateway 2400436 wikitext text/x-wiki tvtxza0fk0dxfzqsqcvwnq96uklk4zz 2400536 2400436 2026-04-10T11:44:52Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2400536 wikitext text/x-wiki m7gj1kk3zmfnmia0bghnicongwfxlj5 2400540 2400536 2026-04-10T11:58:19Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2400540 wikitext text/x-wiki 8xd7qn7hjnedy6yziyb14w7tdegqawr 0 4108 2400058 2400057 2026-04-09T11:59:33Z ScheduleDeploymentBot 37566 Add [[gerrit:1269340]] to Thursday, April 09 UTC afternoon backport window 2400058 wikitext text/x-wiki ix3rb8w6vr8k05t7sqio9lhateicbnc 2400060 2400058 2026-04-09T12:01:57Z Hashar 32 /* {{Deployment_day|date=2026-04-09}} */ 2400060 wikitext text/x-wiki hvrd375gk8q6g0t7fy3k1e988uhjp2w 2400063 2400060 2026-04-09T12:05:06Z ScheduleDeploymentBot 37566 Add [[gerrit:1269345]] to Thursday, April 09 UTC afternoon backport window 2400063 wikitext text/x-wiki hggrpy3os6rujss528tgvomadqf5fmi 2400064 2400063 2026-04-09T12:05:29Z ScheduleDeploymentBot 37566 Add [[gerrit:1269334]] to Thursday, April 09 UTC afternoon backport window 2400064 wikitext text/x-wiki grkv37dhr1p1rirrwyy170tkd1pqpvd 2400187 2400064 2026-04-09T14:23:14Z ScheduleDeploymentBot 37566 Add [[gerrit:1269475]] to Thursday, April 09 UTC late backport window 2400187 wikitext text/x-wiki remk7bii42tlfsk71hydp01s8p6q5gv 2400218 2400187 2026-04-09T15:19:42Z ScheduleDeploymentBot 37566 Add [[gerrit:1047441]] to Thursday, April 09 UTC late backport window 2400218 wikitext text/x-wiki k73z3559ot0fsk24nv0iwl8mjqttjjn 2400262 2400218 2026-04-09T17:15:57Z ScheduleDeploymentBot 37566 Add [[gerrit:1268680]] to Thursday, April 09 UTC late backport window 2400262 wikitext text/x-wiki skj5kyigihxjy64lf52byu9800ws0av 2400288 2400262 2026-04-09T18:17:26Z ScheduleDeploymentBot 37566 Add [[gerrit:1269552]] to Thursday, April 09 UTC late backport window 2400288 wikitext text/x-wiki ht617y2vbm6n5hy1whx9vcnyabopt8e 2400534 2400288 2026-04-10T11:40:02Z ScheduleDeploymentBot 37566 Add [[gerrit:1270016]] to Monday, April 13 UTC afternoon backport window 2400534 wikitext text/x-wiki ffrbg1hqw6woezqhgwgd1m6yzmw8n1q 498 6726 2400156 2399954 2026-04-09T13:41:42Z Stashbot 7414 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2400156 wikitext text/x-wiki deiuqn4p0sw8xcyg4zlg71ifaae9229 2400173 2400156 2026-04-09T13:53:14Z Stashbot 7414 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2400173 wikitext text/x-wiki 458d10xvklrw0u9yj6two3xdant5x7r 2400429 2400173 2026-04-10T08:32:47Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component istio-gateway 2400429 wikitext text/x-wiki qli7mci9ufef722f4dk5swqyimz5fi8 2400430 2400429 2026-04-10T08:32:57Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component istio-gateway 2400430 wikitext text/x-wiki q0n4ssda9pwuvha1rrwphssluz4c1r5 2400433 2400430 2026-04-10T08:41:13Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component istio-gateway 2400433 wikitext text/x-wiki 70uy6lqkbuzxikpojykruaholyfhnqa 2400434 2400433 2026-04-10T08:41:23Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component istio-gateway 2400434 wikitext text/x-wiki 4mq93542jhavf43aq27dr2sw1qtunuy 2400532 2400434 2026-04-10T11:29:40Z Stashbot 7414 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api 2400532 wikitext text/x-wiki hlrjvz2e12dsipjbplp29isoyeq24m6 2400535 2400532 2026-04-10T11:41:14Z Stashbot 7414 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api 2400535 wikitext text/x-wiki 50dvh5b9oh89hnd14fmmgc5efyn35mp 0 7919 2400059 2400054 2026-04-09T12:00:51Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on ms-fe1020.eqiad.wmnet with reason: host reimage 2400059 wikitext text/x-wiki hn4zg0yx1mogxbz6u1l105oaroxd98e 2400061 2400059 2026-04-09T12:02:05Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.dns.netbox 2400061 wikitext text/x-wiki klxqtpt3gzl22v0jylpa2vpw24k993v 2400062 2400061 2026-04-09T12:04:34Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-fe1020.eqiad.wmnet with reason: host reimage 2400062 wikitext text/x-wiki s050tr0aegbitp5re39c0enhcyzejbd 2400065 2400062 2026-04-09T12:05:43Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding apus-be1005 to eqiad - jclark@cumin1003" 2400065 wikitext text/x-wiki 4bxtykpve9ld4q89dozrd5iv9tzf5bz 2400066 2400065 2026-04-09T12:05:49Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding apus-be1005 to eqiad - jclark@cumin1003" 2400066 wikitext text/x-wiki oc1luawp1ql0ts6zi7frtikawltcjen 2400067 2400066 2026-04-09T12:05:50Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2400067 wikitext text/x-wiki lk633rbvi0ffr1ohm6cvb5svkomdae6 2400068 2400067 2026-04-09T12:06:25Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host apus-be1005.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400068 wikitext text/x-wiki 8aso68j6lc3w81lr5vnyxeenbhnzevd 2400069 2400068 2026-04-09T12:06:49Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host apus-be1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400069 wikitext text/x-wiki 8wtyhc8nhezx4rk1csqjadta25ur3yt 2400070 2400069 2026-04-09T12:07:50Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400070 wikitext text/x-wiki tplibntythfpicb58fh8ux5x4vk45gk 2400071 2400070 2026-04-09T12:08:25Z Stashbot 7414 moritzm: restarting Postfix on mx-out to pick up OpenSSL updates 2400071 wikitext text/x-wiki sto23n0bvk7xbi67w7eedwyy63fvkyd 2400072 2400071 2026-04-09T12:09:44Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400072 wikitext text/x-wiki hixle93020ypeyju2zyakpl0eiitiax 2400073 2400072 2026-04-09T12:10:18Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400073 wikitext text/x-wiki 1k6bd7qkyem9hld56nnjyr6wiledytz 2400074 2400073 2026-04-09T12:12:13Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400074 wikitext text/x-wiki dfjyrpwhmws4nkcqxq8qyoikzgjwwo5 2400075 2400074 2026-04-09T12:13:41Z Stashbot 7414 fnegri@cumin1003: conftool action : set/pooled=no; selector: name=clouddb1019.eqiad.wmnet 2400075 wikitext text/x-wiki 3s4ncsmet6td15nt3j54viiavkv5hbb 2400076 2400075 2026-04-09T12:18:07Z Stashbot 7414 moritzm: restarting Postfix on mx-in to pick up OpenSSL updates 2400076 wikitext text/x-wiki 02anqjm1r03gxz6shxspl4sqo8qsu1v 2400077 2400076 2026-04-09T12:23:51Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-fe1020.eqiad.wmnet with OS bullseye 2400077 wikitext text/x-wiki bw3cdb6mnwpl9am8kjc1v9er9h3fber 2400078 2400077 2026-04-09T12:24:05Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on P{ms-fe[1009-1019,1021-1024].eqiad.wmnet} and (A:swift-fe or A:swift-fe-canary or A:swift-fe-codfw or A:swift-fe-eqiad) 2400078 wikitext text/x-wiki hhots1lszllteaq1sdyo3kzsi2x32zf 2400079 2400078 2026-04-09T12:27:10Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host apus-be1005.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400079 wikitext text/x-wiki e4xxrc8y2qi7epj8o111t825sqy4hds 2400080 2400079 2026-04-09T12:28:16Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host apus-be1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400080 wikitext text/x-wiki i380e0pauhvk39vkxlzbai5rks3n6vs 2400081 2400080 2026-04-09T12:30:32Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400081 wikitext text/x-wiki 8wno8ccoajpuwvg4rzjfvgj9zh95lvi 2400082 2400081 2026-04-09T12:30:41Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400082 wikitext text/x-wiki tvhs3uz6ygxp0ewfjfqlhne8p760ujh 2400084 2400082 2026-04-09T12:32:22Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on P{ms-fe[1009-1019,1021-1024].eqiad.wmnet} and (A:swift-fe or A:swift-fe-canary or A:swift-fe-codfw or A:swift-fe-eqiad) 2400084 wikitext text/x-wiki 5ddoom6oifbuye0befpjfxfwsvlr1oi 2400085 2400084 2026-04-09T12:33:11Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.dns.netbox 2400085 wikitext text/x-wiki jgktlphq95ht676tx9ots7j9ll7uowr 2400086 2400085 2026-04-09T12:34:40Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reimage for host thanos-fe1007.eqiad.wmnet with OS bullseye 2400086 wikitext text/x-wiki rxc6lcw9fkegn32uy6u9iyfi4em4zn7 2400087 2400086 2026-04-09T12:35:11Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.move-vlan for host thanos-fe1007 2400087 wikitext text/x-wiki lcmijytjswgtrueb5tinlavdgce4n3x 2400088 2400087 2026-04-09T12:38:26Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400088 wikitext text/x-wiki fjkidl5uenjf3z8ktok4fd1oo858hwy 2400089 2400088 2026-04-09T12:38:27Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.netbox 2400089 wikitext text/x-wiki nez4sxjlnh8p664fiv5ypnpyk4cs2cr 2400090 2400089 2026-04-09T12:40:16Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400090 wikitext text/x-wiki g0drhfq6fzxgdyhw0uxd0ssqow5omrg 2400091 2400090 2026-04-09T12:40:23Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400091 wikitext text/x-wiki 2no3asnwcxmh4x21g4z2j47zxb4ob76 2400092 2400091 2026-04-09T12:42:10Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding phab1006 to eqiad - jclark@cumin1003" 2400092 wikitext text/x-wiki izlihbn8y40oyf56sbr71yt8ume4nqt 2400093 2400092 2026-04-09T12:42:11Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-cluster depool all services in eqiad/aux-eqiad: maintenance 2400093 wikitext text/x-wiki oka81n9klf1pplh21t5ibr3pne5zbfb 2400094 2400093 2026-04-09T12:42:15Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding phab1006 to eqiad - jclark@cumin1003" 2400094 wikitext text/x-wiki s0mcw3ow9598tp7zti0gm1wfzeddtmd 2400095 2400094 2026-04-09T12:42:16Z Stashbot 7414 jclark@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2400095 wikitext text/x-wiki 8x7g89xxa28f5kdxbhl4cxrti1i3hpd 2400096 2400095 2026-04-09T12:42:17Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400096 wikitext text/x-wiki 9ftm0lhreujqopau9tjlm7q9sji1hng 2400097 2400096 2026-04-09T12:42:26Z Stashbot 7414 mvernon@cumin2002: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2400097 wikitext text/x-wiki 66dozdr5voofszrxt488t066dycp6yn 2400098 2400097 2026-04-09T12:42:38Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-cluster (exit_code=0) depool all services in eqiad/aux-eqiad: maintenance 2400098 wikitext text/x-wiki n9juykip1gwlanrdv7wds3q7v0w1ed7 2400099 2400098 2026-04-09T12:42:39Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.netbox 2400099 wikitext text/x-wiki tml7poep8sktk1kl1nqsp173ctnv30u 2400100 2400099 2026-04-09T12:42:43Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host phab1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400100 wikitext text/x-wiki 4fhvhb9dpy5qr90txuk5m6cmco1kgek 2400101 2400100 2026-04-09T12:46:10Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.wipe-cluster Wipe the K8s cluster aux-eqiad: Kubernetes upgrade 2400101 wikitext text/x-wiki qy2a1euzjopyfwhq6928qa4n27yi1rl 2400102 2400101 2026-04-09T12:46:25Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host thanos-fe1007 - mvernon@cumin2002" 2400102 wikitext text/x-wiki 5pqmbckjieomd0ikpq290zgr4vsimhn 2400103 2400102 2026-04-09T12:46:31Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host thanos-fe1007 - mvernon@cumin2002" 2400103 wikitext text/x-wiki ha9nn2dqnxoeq12agrn8v015gvpuggv 2400104 2400103 2026-04-09T12:46:32Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2400104 wikitext text/x-wiki 0uh62pqhhswagwamojmy74pf4ds3bcq 2400105 2400104 2026-04-09T12:46:33Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.wipe-cache thanos-fe1007.eqiad.wmnet 186.48.64.10.in-addr.arpa 6.8.1.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2400105 wikitext text/x-wiki ioel73onvr3gzmzbjsypi5drzet7jcq 2400106 2400105 2026-04-09T12:46:35Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) thanos-fe1007.eqiad.wmnet 186.48.64.10.in-addr.arpa 6.8.1.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2400106 wikitext text/x-wiki oveooxm4gm347x84h0h7pduqhiipq7z 2400107 2400106 2026-04-09T12:46:36Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host thanos-fe1007 2400107 wikitext text/x-wiki 8dyob51tuuxqhdg5a2nfo7qv3zv3os3 2400108 2400107 2026-04-09T12:47:00Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host thanos-fe1007 2400108 wikitext text/x-wiki nvfx4ok2d3cxb11z66zi8x4m26m7hdq 2400109 2400108 2026-04-09T12:47:02Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host thanos-fe1007 2400109 wikitext text/x-wiki tn82eqomd3q24u399x48lamis8gyner 2400110 2400109 2026-04-09T12:53:59Z Stashbot 7414 hashar: Directly pushed GrowthExperiments wmf/1.46.0-wmf.22 patch https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GrowthExperiments/+/1269351 due to a chicken-and-egg issue on that branch 2400110 wikitext text/x-wiki b17a6ey0vt8f6yaj9q4f03nojk1ri22 2400111 2400110 2026-04-09T12:59:03Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/admin 'sync'. 2400111 wikitext text/x-wiki 5j67qyv95a0i6mcpuswwn81fxu78wmi 2400112 2400111 2026-04-09T12:59:26Z Stashbot 7414 jmm@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2400112 wikitext text/x-wiki ltuvskru3cqk1ztf2wk32vbya1l4u2h 2400113 2400112 2026-04-09T12:59:45Z Stashbot 7414 jmm@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2400113 wikitext text/x-wiki 0upx2f8a21gohueqvfoo9q3chxoflmu 2400114 2400113 2026-04-09T13:01:17Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'sync'. 2400114 wikitext text/x-wiki cqparzqssfkgyfzou86kp1v7nt4nsea 2400115 2400114 2026-04-09T13:01:38Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host phab1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400115 wikitext text/x-wiki pj1j8k4sa94e7ygmjguzkxarh4jjy7t 2400116 2400115 2026-04-09T13:02:48Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on thanos-fe1007.eqiad.wmnet with reason: host reimage 2400116 wikitext text/x-wiki m9u1gqfyl68jvjoohexb6v6vpee47wu 2400117 2400116 2026-04-09T13:03:07Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2400117 wikitext text/x-wiki q6voi12ix9z97yn6r3dwph9wq8ajqbd 2400118 2400117 2026-04-09T13:04:00Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host phab1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400118 wikitext text/x-wiki oj718a0a1qgi0d2r5ckpc7j33udgz08 2400119 2400118 2026-04-09T13:05:05Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/jaeger: sync 2400119 wikitext text/x-wiki 9ayoj6zhsandjnzraz3ygslfuz1x0rz 2400120 2400119 2026-04-09T13:05:16Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/jaeger: sync 2400120 wikitext text/x-wiki k010sf1anud8vev3fsgymkudpp9wfc0 2400121 2400120 2026-04-09T13:05:37Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/kafka-mirrormaker: sync 2400121 wikitext text/x-wiki qfmp7k907xf03odyinj4brg7eq6f31a 2400122 2400121 2026-04-09T13:05:45Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/kafka-mirrormaker: sync 2400122 wikitext text/x-wiki ju5wn24x4v3qowaigvx47gsls6dzj9d 2400123 2400122 2026-04-09T13:05:56Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/redioscope: sync 2400123 wikitext text/x-wiki 5psu6f0jfkwkbx5uf81gw1pyy7etiq0 2400124 2400123 2026-04-09T13:06:04Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/redioscope: sync 2400124 wikitext text/x-wiki 474zsystz6p366cxrwfe02g5r4gtyxk 2400125 2400124 2026-04-09T13:06:11Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/sophroid: sync 2400125 wikitext text/x-wiki f2g2de9p0mi0ds198tfzv1ss2tk3dg6 2400126 2400125 2026-04-09T13:06:19Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/sophroid: sync 2400126 wikitext text/x-wiki myt5ikmvze0g659bghs6le2m0nf5dte 2400127 2400126 2026-04-09T13:06:33Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2400127 wikitext text/x-wiki 2f6gysyeck5beuqy5u1bsgigail6zmg 2400128 2400127 2026-04-09T13:07:13Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/services/miscweb: sync 2400128 wikitext text/x-wiki peirp378lef6bc7nl7tvt7h7kqaxrou 2400129 2400128 2026-04-09T13:07:19Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host phab1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400129 wikitext text/x-wiki lzh5lv2w5s675ov6hxzxohvr132733r 2400130 2400129 2026-04-09T13:07:28Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/services/miscweb: sync 2400130 wikitext text/x-wiki 2pi3mxzbkigeihnnqihvuesa0cbmjft 2400131 2400130 2026-04-09T13:07:38Z Stashbot 7414 jclark@cumin1003: START - Cookbook sre.hosts.provision for host phab1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400131 wikitext text/x-wiki rlync7ws0dxkc7q5ai4c257nnamxycm 2400132 2400131 2026-04-09T13:08:41Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2400132 wikitext text/x-wiki jn2obggzprw6p7enqpmwg4xe7pqz9jv 2400133 2400132 2026-04-09T13:08:45Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.wipe-cluster (exit_code=0) Wipe the K8s cluster aux-eqiad: Kubernetes upgrade 2400133 wikitext text/x-wiki hm2y3ytp2wlt23szcrz1xwb35lnrroc 2400134 2400133 2026-04-09T13:08:48Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on thanos-fe1007.eqiad.wmnet with reason: host reimage 2400134 wikitext text/x-wiki 3wj763o6zdtz43cyphecfjyj6w6aia6 2400136 2400134 2026-04-09T13:09:07Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2400136 wikitext text/x-wiki et6c1den63a0az7b50kqev3nbtf5zj6 2400137 2400136 2026-04-09T13:09:31Z Stashbot 7414 elukey@cumin1003: START - Cookbook sre.k8s.pool-depool-cluster pool all services in eqiad/aux-eqiad: maintenance 2400137 wikitext text/x-wiki anubp3his09y6vobsyf43chwxwvogc7 2400138 2400137 2026-04-09T13:09:55Z Stashbot 7414 elukey@cumin1003: END (PASS) - Cookbook sre.k8s.pool-depool-cluster (exit_code=0) pool all services in eqiad/aux-eqiad: maintenance 2400138 wikitext text/x-wiki m3nw3ro6xbms49wdkeullk2ws4jwbzy 2400139 2400138 2026-04-09T13:14:52Z Stashbot 7414 jclark@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host phab1006.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400139 wikitext text/x-wiki 7hrzgj10gjryo5xp9nwv49vi1r76n5u 2400140 2400139 2026-04-09T13:15:40Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2400140 wikitext text/x-wiki rly1avx5pgzn6xurmeorbfvvv3yo7nc 2400141 2400140 2026-04-09T13:17:12Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2400141 wikitext text/x-wiki gsoqgafr87ul5vmub79z3q0rnykdk6a 2400142 2400141 2026-04-09T13:19:17Z Stashbot 7414 fceratto@deploy1003: helmfile [aux-k8s-eqiad] 'sync' command on namespace 'zarcillo' for release 'main' . 2400142 wikitext text/x-wiki it0ob0s86ih29mdgt880gfoy0bdsoi9 2400143 2400142 2026-04-09T13:27:50Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host thanos-fe1007.eqiad.wmnet with OS bullseye 2400143 wikitext text/x-wiki glto938x6r4svswoa9t7fp54pyeqq7r 2400144 2400143 2026-04-09T13:28:58Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] START helmfile.d/dse-k8s-services/opensearch-ipoid: apply 2400144 wikitext text/x-wiki 0opnjgv3pst2hh4n2czrpf9htew6ezz 2400145 2400144 2026-04-09T13:29:08Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies rolling restart_daemons on P{thanos-fe[1004-1006].eqiad.wmnet} and (A:thanos-fe or A:thanos-fe-codfw or A:thanos-fe-eqiad) 2400145 wikitext text/x-wiki krt0dwt7k2ozvwoj2wwr9zri5gdvj5g 2400146 2400145 2026-04-09T13:30:49Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies (exit_code=0) rolling restart_daemons on P{thanos-fe[1004-1006].eqiad.wmnet} and (A:thanos-fe or A:thanos-fe-codfw or A:thanos-fe-eqiad) 2400146 wikitext text/x-wiki s82ni1wbttxhss5kie4f3x6fpby34h6 2400147 2400146 2026-04-09T13:31:13Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies rolling restart_daemons on A:thanos-fe-codfw 2400147 wikitext text/x-wiki 0z8czc7b7keaiygd1qga1qgl5mtesgd 2400148 2400147 2026-04-09T13:33:16Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies (exit_code=0) rolling restart_daemons on A:thanos-fe-codfw 2400148 wikitext text/x-wiki qmqm5hpwoguq7sw8cu6x035tfjbfa7z 2400149 2400148 2026-04-09T13:33:24Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-codfw] DONE helmfile.d/dse-k8s-services/opensearch-ipoid: apply 2400149 wikitext text/x-wiki 63eh4tgg61fd01va6os899tqfest4cg 2400150 2400149 2026-04-09T13:36:39Z Stashbot 7414 hashar@deploy1003: Started scap sync-world: Backport for [[gerrit:1269351|fix: adjust to return type changed by upstream]], [[gerrit:1269339|browser-tests: hide Cypress tests from CI (T419574)]], [[gerrit:1269340|browser-tests: hide Cypress tests from CI (T419574)]], [[gerrit:1269345|Fix BackfillInterwikiRightsLog wrt. cyclic renames (T6055)]], [[gerrit:1269334|Fix BackfillInterwikiRightsLog wrt. cyclic renames (T6055 2400150 wikitext text/x-wiki 5vp9ahffalvjidzs03ou6k3xjmqhlae 2400151 2400150 2026-04-09T13:37:18Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2400151 wikitext text/x-wiki h37pq80bup7oyjirnwzyurs58f7qado 2400152 2400151 2026-04-09T13:38:36Z Stashbot 7414 hashar@deploy1003: mszwarc, hashar: Backport for [[gerrit:1269351|fix: adjust to return type changed by upstream]], [[gerrit:1269339|browser-tests: hide Cypress tests from CI (T419574)]], [[gerrit:1269340|browser-tests: hide Cypress tests from CI (T419574)]], [[gerrit:1269345|Fix BackfillInterwikiRightsLog wrt. cyclic renames (T6055)]], [[gerrit:1269334|Fix BackfillInterwikiRightsLog wrt. cyclic renames (T6055)]] sync 2400152 wikitext text/x-wiki eihm3uxkiroj1ayckoktur8q06g9n9b 2400153 2400152 2026-04-09T13:39:34Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reimage for host apus-fe1003.eqiad.wmnet with OS bookworm 2400153 wikitext text/x-wiki dv8o972kesoc55wmmqfsg25u2c514x8 2400154 2400153 2026-04-09T13:40:08Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.move-vlan for host apus-fe1003 2400154 wikitext text/x-wiki 851c0ytpdgyzmm76jvo5ot5zpgnwj3h 2400155 2400154 2026-04-09T13:40:17Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.netbox 2400155 wikitext text/x-wiki kwad5wwisbbhpd12t80gpgmp37qucbv 2400157 2400155 2026-04-09T13:44:18Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply 2400157 wikitext text/x-wiki d3frc4g40oftmsg5h1gkkupsjpfw2te 2400158 2400157 2026-04-09T13:44:23Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply 2400158 wikitext text/x-wiki n5bmjie65ozramn0pscgdfkhuob91qj 2400159 2400158 2026-04-09T13:44:30Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host apus-fe1003 - mvernon@cumin2002" 2400159 wikitext text/x-wiki jrlxe16ts8589eyyzlwvko4q6p25e43 2400160 2400159 2026-04-09T13:44:35Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host apus-fe1003 - mvernon@cumin2002" 2400160 wikitext text/x-wiki 8s7gz3qcf2m49rookdsrz9uzctbmsow 2400161 2400160 2026-04-09T13:44:37Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2400161 wikitext text/x-wiki 6rdwxlikiygyocgtnglpzxgfzaco9mo 2400162 2400161 2026-04-09T13:44:38Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.wipe-cache apus-fe1003.eqiad.wmnet 102.32.64.10.in-addr.arpa 2.0.1.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2400162 wikitext text/x-wiki p6q4mx9ncxrk8s3xnyar2h8ftimv1dj 2400163 2400162 2026-04-09T13:44:40Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) apus-fe1003.eqiad.wmnet 102.32.64.10.in-addr.arpa 2.0.1.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2400163 wikitext text/x-wiki 074xoyca1esdvmoj5x0w5c4msyl8224 2400164 2400163 2026-04-09T13:44:41Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host apus-fe1003 2400164 wikitext text/x-wiki 9kbw6i6pbzs411eduzbth76ev6ksfk7 2400165 2400164 2026-04-09T13:44:59Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host apus-fe1003 2400165 wikitext text/x-wiki 6q8tn8lnvym0klskodj61cv1s6mdolo 2400166 2400165 2026-04-09T13:45:00Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host apus-fe1003 2400166 wikitext text/x-wiki qvev4uabn8pe4lq3dz1x39e0drl9uzl 2400167 2400166 2026-04-09T13:45:35Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/opensearch-ipoid: apply 2400167 wikitext text/x-wiki 1nvvho61g6k0iirg9sjjdzuzqeu2zj8 2400168 2400167 2026-04-09T13:46:50Z Stashbot 7414 bking@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/opensearch-ipoid: apply 2400168 wikitext text/x-wiki 6djn06svcgzijjvj1v9vdpswp33p35x 2400169 2400168 2026-04-09T13:48:14Z Stashbot 7414 hashar@deploy1003: mszwarc, hashar: Continuing with sync 2400169 wikitext text/x-wiki ged0mk3guew6kf2khhv138lf2311i5w 2400170 2400169 2026-04-09T13:52:19Z Stashbot 7414 hashar@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269351|fix: adjust to return type changed by upstream]], [[gerrit:1269339|browser-tests: hide Cypress tests from CI (T419574)]], [[gerrit:1269340|browser-tests: hide Cypress tests from CI (T419574)]], [[gerrit:1269345|Fix BackfillInterwikiRightsLog wrt. cyclic renames (T6055)]], [[gerrit:1269334|Fix BackfillInterwikiRightsLog wrt. cyclic renames (T605 2400170 wikitext text/x-wiki dywf94h7hnn6g1f6zey9iop05v6l28k 2400171 2400170 2026-04-09T13:52:45Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400171 wikitext text/x-wiki dwcp0zzvlix7p9l1ap9dwytg5ho3yzb 2400172 2400171 2026-04-09T13:52:55Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400172 wikitext text/x-wiki eysct99ouo8kaq9tsq6ipbzvnk7509f 2400175 2400172 2026-04-09T13:55:44Z Stashbot 7414 aude@deploy1003: Started scap sync-world: Backport for [[gerrit:1269063|Enable reading list beta feature for pilot wikis (T420878)]] 2400175 wikitext text/x-wiki alequ6o93r7n9nmg1s8b7u1h5dzc6sr 2400176 2400175 2026-04-09T13:57:36Z Stashbot 7414 aude@deploy1003: bwang, aude: Backport for [[gerrit:1269063|Enable reading list beta feature for pilot wikis (T420878)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400176 wikitext text/x-wiki blb7d1ise6azk5ozqn3hyhbqlzyj57b 2400177 2400176 2026-04-09T14:00:07Z Stashbot 7414 aude@deploy1003: bwang, aude: Continuing with sync 2400177 wikitext text/x-wiki 67xxwcfsem9f6qkkie1qjlpjcxiwxfd 2400178 2400177 2026-04-09T14:01:44Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on apus-fe1003.eqiad.wmnet with reason: host reimage 2400178 wikitext text/x-wiki sviiy5e0l0f1kn8ogybb70w9kloidjp 2400179 2400178 2026-04-09T14:04:26Z Stashbot 7414 aude@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269063|Enable reading list beta feature for pilot wikis (T420878)]] (duration: 08m 40s) 2400179 wikitext text/x-wiki tpxmyepy465ochglfr4hazydkhuo9di 2400180 2400179 2026-04-09T14:06:34Z Stashbot 7414 mszwarc@deploy1003: mwscript-k8s job started: foreachwikiindblist all backfillInterwikiRightsLog.php --remote-wiki metawiki 20260311190000 # T6055 (second attempt) 2400180 wikitext text/x-wiki kg5r21l25qziiglucq1lr7qj1ei1i31 2400182 2400180 2026-04-09T14:09:33Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on apus-fe1003.eqiad.wmnet with reason: host reimage 2400182 wikitext text/x-wiki 9eh6an559u2f5mwd1t4bvfg2aejk6ad 2400183 2400182 2026-04-09T14:13:28Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2400183 wikitext text/x-wiki t55rs1i2ng9kirj3abstttq9fqajixf 2400184 2400183 2026-04-09T14:14:03Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2400184 wikitext text/x-wiki 3gnst15eo6jtbaaevv46r0cbxwfgs39 2400185 2400184 2026-04-09T14:14:33Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2400185 wikitext text/x-wiki 108egujg14fyzn3bpzhtrv65jntcmfg 2400189 2400185 2026-04-09T14:23:30Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400189 wikitext text/x-wiki k9emjsgqat08nadzxrxelrjxmj7wagb 2400190 2400189 2026-04-09T14:23:45Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400190 wikitext text/x-wiki dfjtcdlf4lxtnrpx7pkfb9k9eq5mu8x 2400191 2400190 2026-04-09T14:25:48Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400191 wikitext text/x-wiki nu4hbysu2j08dus5ew3tqje0hx4391g 2400192 2400191 2026-04-09T14:25:54Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400192 wikitext text/x-wiki kknlpoi0orng9ydpmyfyyum16r7istx 2400193 2400192 2026-04-09T14:26:30Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2400193 wikitext text/x-wiki qcmhdrqe7810wu4ozejbh168aij6qv6 2400194 2400193 2026-04-09T14:29:46Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400194 wikitext text/x-wiki dkz5kaz52od36kg35w2raq75wzgrukf 2400195 2400194 2026-04-09T14:29:53Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400195 wikitext text/x-wiki ceaoc77bt1wa78fxy3d9y7ispkl6ah5 2400196 2400195 2026-04-09T14:30:50Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host apus-fe1003.eqiad.wmnet with OS bookworm 2400196 wikitext text/x-wiki rwpgmk7vpxvg3bxo0ipf3ezisxr5j5k 2400197 2400196 2026-04-09T14:35:40Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400197 wikitext text/x-wiki 7hhq4fm0f6c1u6gjxkodcg77u1gh6lm 2400198 2400197 2026-04-09T14:35:52Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400198 wikitext text/x-wiki 5gwkhqvi5l95nau4xdpnmmcj9g6h5oj 2400199 2400198 2026-04-09T14:37:38Z Stashbot 7414 Emperor: ceph orch host drain moss-be1002 --zap-osd-devices T421719 2400199 wikitext text/x-wiki a7pbjdkvz1w9ughwe4nmec3hq4vne2b 2400200 2400199 2026-04-09T14:38:54Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400200 wikitext text/x-wiki 4264scxd89mk87ezje49r64nl6nyn93 2400201 2400200 2026-04-09T14:39:53Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.dns.roll-restart-reboot-wikimedia-dns rolling restart_daemons on A:wikidough 2400201 wikitext text/x-wiki gg07d7d67tmukml95q5eji9cjle3dzw 2400202 2400201 2026-04-09T14:40:46Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400202 wikitext text/x-wiki bpjqfmhagoujw8zbdx0ugatob4gtc1j 2400203 2400202 2026-04-09T14:41:06Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400203 wikitext text/x-wiki eezsznpfn5nw0nha4xtrch1zp4nc4u3 2400204 2400203 2026-04-09T14:41:13Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400204 wikitext text/x-wiki smll79vo8edc0e047foxnqzb894sfxw 2400205 2400204 2026-04-09T14:42:26Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400205 wikitext text/x-wiki tgtttw00iapmy88qnxhqhhqfsf4g7qx 2400206 2400205 2026-04-09T14:44:20Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400206 wikitext text/x-wiki 6wf8e99oenqzsnfwvv62gtqeqfv88vo 2400207 2400206 2026-04-09T14:45:29Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2400207 wikitext text/x-wiki qnat6hq9qfyhz7ex2umyrzwy4lqvjf0 2400208 2400207 2026-04-09T14:47:44Z Stashbot 7414 sukhe@cumin1003: START - Cookbook sre.dns.roll-restart rolling restart_daemons on A:dnsbox and (A:dnsbox) 2400208 wikitext text/x-wiki gu3pgmrk9a7vgutso413csvyzbs1n3y 2400209 2400208 2026-04-09T14:53:23Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.dns.roll-restart-reboot-wikimedia-dns (exit_code=0) rolling restart_daemons on A:wikidough 2400209 wikitext text/x-wiki bo2qa2o9q57gb1o15fb01relt5lxnuw 2400210 2400209 2026-04-09T14:58:04Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.245.0" for 2 host(s) 2400210 wikitext text/x-wiki c74o3y18wqpya6ea9tc426s4ddjt3w0 2400212 2400210 2026-04-09T14:59:56Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.245.0" completed for 2 hosts 2400212 wikitext text/x-wiki iqtsoowz91q1mnmym2wto7k2b83uzyr 2400213 2400212 2026-04-09T15:03:24Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revertrisk' for release 'main' . 2400213 wikitext text/x-wiki nmq5jrmzsnoejr8ad11q66vc4pq81uv 2400215 2400213 2026-04-09T15:17:39Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400215 wikitext text/x-wiki rduvul677d8jxn0nul0kwd8xsbu472a 2400216 2400215 2026-04-09T15:19:32Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400216 wikitext text/x-wiki 31lob6ffb1c12km707vuf89abuyqx37 2400217 2400216 2026-04-09T15:19:41Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400217 wikitext text/x-wiki 0oy0j6yxb2qyfpb1e380pzbukmt0yjx 2400219 2400217 2026-04-09T15:19:47Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.reimage for host moss-be1002.eqiad.wmnet with OS bookworm 2400219 wikitext text/x-wiki f2fb1es403bcn48be46x2ktfewt6o2f 2400220 2400219 2026-04-09T15:20:09Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.move-vlan for host moss-be1002 2400220 wikitext text/x-wiki i9l6af74f8roc7whada7eibr35notf8 2400221 2400220 2026-04-09T15:20:19Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.netbox 2400221 wikitext text/x-wiki hzkbnfc22iocikjqaqqe4rhsyr4knsq 2400222 2400221 2026-04-09T15:21:33Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400222 wikitext text/x-wiki g9in8yjalbrffbfxu40kcpqmnn8k8n7 2400223 2400222 2026-04-09T15:25:25Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host moss-be1002 - mvernon@cumin2002" 2400223 wikitext text/x-wiki atlnh783tm01o7isgw7r7i7kgxdo7ad 2400224 2400223 2026-04-09T15:25:30Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host moss-be1002 - mvernon@cumin2002" 2400224 wikitext text/x-wiki notm990lnevfxcs5fb1p3gxw161kk7y 2400225 2400224 2026-04-09T15:25:31Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2400225 wikitext text/x-wiki g1pya3e1bl9cqlkgq5b55sn1vctbfdk 2400226 2400225 2026-04-09T15:25:32Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.dns.wipe-cache moss-be1002.eqiad.wmnet 79.32.64.10.in-addr.arpa 9.7.0.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2400226 wikitext text/x-wiki dx10e1spb5d7w8q175o4zs7elf39jpl 2400227 2400226 2026-04-09T15:25:35Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) moss-be1002.eqiad.wmnet 79.32.64.10.in-addr.arpa 9.7.0.0.2.3.0.0.4.6.0.0.0.1.0.0.3.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2400227 wikitext text/x-wiki c7uh6bze6m8098jwgoikuo8hnvnvq4j 2400228 2400227 2026-04-09T15:25:36Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host moss-be1002 2400228 wikitext text/x-wiki hmbahcuuzvi8q58e6li860yc7jtiggh 2400229 2400228 2026-04-09T15:25:59Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host moss-be1002 2400229 wikitext text/x-wiki ehltgx9qzzcrp2q46u1ejivbin4vbwd 2400230 2400229 2026-04-09T15:26:00Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host moss-be1002 2400230 wikitext text/x-wiki begly3hb1nwwq7tbzdnm5cdc98y1nbz 2400231 2400230 2026-04-09T15:31:02Z Stashbot 7414 cgoubert@deploy1003: Started scap sync-world: swift service proxy configuration cahnges 2400231 wikitext text/x-wiki 6dt71bkuf41tu6mvnnhfbv6uouv71o0 2400232 2400231 2026-04-09T15:36:07Z Stashbot 7414 cgoubert@deploy1003: Finished scap sync-world: swift service proxy configuration cahnges (duration: 05m 45s) 2400232 wikitext text/x-wiki jzfe0ci73ukibm3756kth04ibgw4i2p 2400233 2400232 2026-04-09T15:42:57Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-experimental: apply 2400233 wikitext text/x-wiki 8jm1kpsbj4s14ifu2frfcqv760fbu9v 2400234 2400233 2026-04-09T15:43:24Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply 2400234 wikitext text/x-wiki pg0pkb6mv29nipb5r89j854buywfx0w 2400235 2400234 2026-04-09T15:43:33Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/mw-experimental: apply 2400235 wikitext text/x-wiki owb2zqx17c7ha9xcn75s5edu4fm2nic 2400236 2400235 2026-04-09T15:44:04Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-experimental: apply 2400236 wikitext text/x-wiki riazjq2bnpbtjjyl4swyd0kvxr6txea 2400237 2400236 2026-04-09T15:44:10Z Stashbot 7414 mvernon@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on moss-be1002.eqiad.wmnet with reason: host reimage 2400237 wikitext text/x-wiki eahlk2ckax81ej0b6yy6afc1tw1xkdm 2400238 2400237 2026-04-09T15:46:40Z Stashbot 7414 sukhe@cumin1003: END (PASS) - Cookbook sre.dns.roll-restart (exit_code=0) rolling restart_daemons on A:dnsbox and (A:dnsbox) 2400238 wikitext text/x-wiki jnfsc4fjcbtq7kpjisaupdox9h7qfob 2400239 2400238 2026-04-09T15:47:56Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on moss-be1002.eqiad.wmnet with reason: host reimage 2400239 wikitext text/x-wiki ppqclnn7e1v3elbd6wfp4kb7dsb7755 2400240 2400239 2026-04-09T16:11:20Z Stashbot 7414 mvernon@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host moss-be1002.eqiad.wmnet with OS bookworm 2400240 wikitext text/x-wiki n7bj0o1w1mu2cnuzx81dkdkw8tigc0s 2400241 2400240 2026-04-09T16:41:35Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1269524|Revert^2 "Use envoy for swift inside mediawiki" (T328872)]] 2400241 wikitext text/x-wiki 3p0p8o5i3jb1iso4kj51imptj2m83i8 2400242 2400241 2026-04-09T16:43:32Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1269524|Revert^2 "Use envoy for swift inside mediawiki" (T328872)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400242 wikitext text/x-wiki p41wmqe4sfem9f0zzkc27sks696c1by 2400243 2400242 2026-04-09T16:44:48Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with sync 2400243 wikitext text/x-wiki 0vuuz01qmlp0sw25nm58r3zeeeoczp1 2400244 2400243 2026-04-09T16:46:56Z Stashbot 7414 aokoth@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: Security Release - T422668 2400244 wikitext text/x-wiki oh69cx9f06gnvqj7368p8x6894mosd5 2400245 2400244 2026-04-09T16:48:38Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269524|Revert^2 "Use envoy for swift inside mediawiki" (T328872)]] (duration: 07m 02s) 2400245 wikitext text/x-wiki 8tf2oc6cb4ap293rfn0gfhmc1d8wg8t 2400246 2400245 2026-04-09T16:51:50Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1269524|Revert^2 "Use envoy for swift inside mediawiki" (T328872)]] 2400246 wikitext text/x-wiki s2yklwq5gdpkje79ewgyyoe5vc0qtlc 2400248 2400246 2026-04-09T16:56:08Z Stashbot 7414 aokoth@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1003.wikimedia.org with reason: Security Release - T422668 2400248 wikitext text/x-wiki 2kia4h61lbz16sv1ayflrmqfh8v1930 2400249 2400248 2026-04-09T16:57:27Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1269532|Revert^3 "Use envoy for swift inside mediawiki"]] 2400249 wikitext text/x-wiki 76n6cdwpprgwpp0voyztyg086i04mme 2400250 2400249 2026-04-09T16:59:20Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1269532|Revert^3 "Use envoy for swift inside mediawiki"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400250 wikitext text/x-wiki mofwdugtpx31liewm5d603na7qjsud7 2400252 2400250 2026-04-09T16:59:45Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with sync 2400252 wikitext text/x-wiki lp09guayrkjf0u9zcebqo7gxbdfsppr 2400253 2400252 2026-04-09T17:01:14Z Stashbot 7414 aokoth@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab2002.wikimedia.org with reason: Security Release - T422668 2400253 wikitext text/x-wiki f7il2dufnjge9fk92rxl83b1n58e14u 2400254 2400253 2026-04-09T17:03:38Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269532|Revert^3 "Use envoy for swift inside mediawiki"]] (duration: 06m 11s) 2400254 wikitext text/x-wiki g9ehhtn1jj2rtrb6lpof3rsncd0d9jp 2400255 2400254 2026-04-09T17:07:47Z Stashbot 7414 bd808@deploy1003: helmfile [staging] START helmfile.d/services/developer-portal: apply 2400255 wikitext text/x-wiki 7bta7soavfxq3xckd53lqdn9ogt1xit 2400256 2400255 2026-04-09T17:08:30Z Stashbot 7414 bd808@deploy1003: helmfile [staging] DONE helmfile.d/services/developer-portal: apply 2400256 wikitext text/x-wiki 32kz9yfvs3xq7uyjlyiu339c3fd8et3 2400257 2400256 2026-04-09T17:08:44Z Stashbot 7414 bd808@deploy1003: helmfile [codfw] START helmfile.d/services/developer-portal: apply 2400257 wikitext text/x-wiki 39rnu60dr7bus8cmhe48soae4xzrt3p 2400258 2400257 2026-04-09T17:09:00Z Stashbot 7414 bd808@deploy1003: helmfile [codfw] DONE helmfile.d/services/developer-portal: apply 2400258 wikitext text/x-wiki laq4pwu5zf60de3niei7nz0wfmiw90q 2400259 2400258 2026-04-09T17:09:11Z Stashbot 7414 bd808@deploy1003: helmfile [eqiad] START helmfile.d/services/developer-portal: apply 2400259 wikitext text/x-wiki oztchanip6i16vixgk4cmhrdf2ff9bb 2400260 2400259 2026-04-09T17:09:33Z Stashbot 7414 bd808@deploy1003: helmfile [eqiad] DONE helmfile.d/services/developer-portal: apply 2400260 wikitext text/x-wiki 3x63le12a3e76jl5v2swrgln1p8mhh1 2400261 2400260 2026-04-09T17:10:58Z Stashbot 7414 aokoth@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab2002.wikimedia.org with reason: Security Release - T422668 2400261 wikitext text/x-wiki 6e5d8k5lvyqtzfepgrpdjr1l10366kh 2400263 2400261 2026-04-09T17:23:27Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/shellbox-video: apply 2400263 wikitext text/x-wiki 8qqrsuu9vbfdc9nnozx6ib6g5ti4lzz 2400264 2400263 2026-04-09T17:24:10Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/shellbox-video: apply 2400264 wikitext text/x-wiki ewtm39brkpl51a2yi82yvj8mgv8iw6d 2400265 2400264 2026-04-09T17:24:19Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/shellbox-video: apply 2400265 wikitext text/x-wiki os4x853m8ldwn929vmzxtdpx0m55660 2400266 2400265 2026-04-09T17:25:05Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/shellbox-video: apply 2400266 wikitext text/x-wiki bfarxq2br39eoq379r9b57sk7018049 2400268 2400266 2026-04-09T17:35:43Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/shellbox-video: apply 2400268 wikitext text/x-wiki c77o94bcawcg8ym4n4vz2c7jo9df6kw 2400269 2400268 2026-04-09T17:35:51Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/shellbox-video: apply 2400269 wikitext text/x-wiki bq3mpnfv3y6nb63dzb27zgjx8sv8bs2 2400270 2400269 2026-04-09T17:36:19Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1269542|Revert^4 "Use envoy for swift inside mediawiki"]] 2400270 wikitext text/x-wiki ofibfihjp5i0ai80p9bsk4x08mfwyui 2400271 2400270 2026-04-09T17:38:10Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1269542|Revert^4 "Use envoy for swift inside mediawiki"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400271 wikitext text/x-wiki fcnufd3wqz90tuyuaal9tntba9wutg9 2400272 2400271 2026-04-09T17:39:18Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with sync 2400272 wikitext text/x-wiki 0exg5lq45fgso9ajcsbgbwr5dncap9r 2400273 2400272 2026-04-09T17:43:09Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269542|Revert^4 "Use envoy for swift inside mediawiki"]] (duration: 06m 49s) 2400273 wikitext text/x-wiki tdau8sdij88ilyfairhbr7m172zxzm2 2400274 2400273 2026-04-09T17:51:18Z Stashbot 7414 dzahn@deploy1003: helmfile [staging] START helmfile.d/services/miscweb: apply 2400274 wikitext text/x-wiki pk5mk5mit6r9zubfov92v8jy18d3ljt 2400275 2400274 2026-04-09T17:51:35Z Stashbot 7414 dzahn@deploy1003: helmfile [staging] DONE helmfile.d/services/miscweb: apply 2400275 wikitext text/x-wiki 77tlv3ntmeavjqg5ccflt8b0edk6uro 2400276 2400275 2026-04-09T17:51:54Z Stashbot 7414 dzahn@deploy1003: helmfile [codfw] START helmfile.d/services/miscweb: apply 2400276 wikitext text/x-wiki 6uv37ukl2ts8q9tcq9sj6bnl0w7062h 2400277 2400276 2026-04-09T17:52:17Z Stashbot 7414 dzahn@deploy1003: helmfile [codfw] DONE helmfile.d/services/miscweb: apply 2400277 wikitext text/x-wiki heb1uvcbhnl0voxfw7izjhw4tpgwwo5 2400278 2400277 2026-04-09T17:52:31Z Stashbot 7414 dzahn@deploy1003: helmfile [eqiad] START helmfile.d/services/miscweb: apply 2400278 wikitext text/x-wiki tjxb62g888f8c5m5m8r7m8u25wqal9v 2400279 2400278 2026-04-09T17:52:47Z Stashbot 7414 dzahn@deploy1003: helmfile [eqiad] DONE helmfile.d/services/miscweb: apply 2400279 wikitext text/x-wiki 78sy3isc1nchh1qs7izsaflx4z536dl 2400280 2400279 2026-04-09T18:02:00Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.246.0" for 2 host(s) 2400280 wikitext text/x-wiki rh94b76jfvkwsm6ae1z5tjg2ua3k82b 2400281 2400280 2026-04-09T18:03:52Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.246.0" completed for 2 hosts 2400281 wikitext text/x-wiki fsx6r1yh41yo91s8i41g27v314pwdvs 2400282 2400281 2026-04-09T18:04:48Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400282 wikitext text/x-wiki g2th52u4rxp6ygjhb02ewfkh9x2zery 2400283 2400282 2026-04-09T18:04:52Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400283 wikitext text/x-wiki 85a7apsf1t0vd1us8l0ae8zn93u5inj 2400284 2400283 2026-04-09T18:13:34Z Stashbot 7414 dancy@deploy1003: rebuilt and synchronized wikiversions files: group2 to 1.46.0-wmf.23 refs T420481 2400284 wikitext text/x-wiki q2vfsfmo4kmhc23kvnhyt58ntf1e8ji 2400287 2400284 2026-04-09T18:17:22Z Stashbot 7414 andrew@cumin2002: START - Cookbook sre.hosts.reboot-single for host cloudcephmon2005-dev.codfw.wmnet 2400287 wikitext text/x-wiki bwjresp3ghdg7zi00hxm8hmwlgl7pgy 2400294 2400287 2026-04-09T18:23:26Z Stashbot 7414 andrew@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcephmon2005-dev.codfw.wmnet 2400294 wikitext text/x-wiki 9nl7pczszol76aejfx0e59ubp6tcoxx 2400295 2400294 2026-04-09T18:29:16Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400295 wikitext text/x-wiki 3212duysqmz1jicdbh4wyofs9gd882h 2400296 2400295 2026-04-09T18:29:19Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400296 wikitext text/x-wiki 0dss18oo85eo994kia9x8vtw660bfw4 2400299 2400296 2026-04-09T18:37:23Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400299 wikitext text/x-wiki 9bblslmh0q4v718leju9gsxiuco5e9t 2400300 2400299 2026-04-09T18:37:26Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400300 wikitext text/x-wiki jsjgkwga6ihbn56ggh4zu7yhd66qjpg 2400301 2400300 2026-04-09T18:40:16Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400301 wikitext text/x-wiki mnmweurvh5ykcsyxxlicex6v1w3rgtp 2400302 2400301 2026-04-09T18:40:20Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400302 wikitext text/x-wiki cpot5l1xoub8cbj6cbcheshjffr0owo 2400303 2400302 2026-04-09T18:46:54Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400303 wikitext text/x-wiki 99su8ak0x9g4b6lw5yv0h45vfoednis 2400304 2400303 2026-04-09T18:46:58Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400304 wikitext text/x-wiki mrr1c41ttyc5qu69mugz1hgrnlskr8m 2400305 2400304 2026-04-09T18:47:58Z Stashbot 7414 dancy@deploy1003: Installing scap version "4.247.0" for 2 host(s) 2400305 wikitext text/x-wiki b7776n5728r4mdhdrk7smc7aw2trhfv 2400306 2400305 2026-04-09T18:49:21Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400306 wikitext text/x-wiki 7au6m6maoimhrxs3u42r5vdnb9r35xn 2400307 2400306 2026-04-09T18:49:25Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400307 wikitext text/x-wiki ify8rme2e4qezv5q1vctvlitk1ild7d 2400308 2400307 2026-04-09T18:49:49Z Stashbot 7414 dancy@deploy1003: Installation of scap version "4.247.0" completed for 2 hosts 2400308 wikitext text/x-wiki 0r34fn4n7uzymenvpoe6mfbistvotyg 2400309 2400308 2026-04-09T18:50:50Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1269572|Revert^5 "Use envoy for swift inside mediawiki"]] 2400309 wikitext text/x-wiki jwurf8rmdhv19qq6tep4qu25qk8komc 2400310 2400309 2026-04-09T18:52:36Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1269572|Revert^5 "Use envoy for swift inside mediawiki"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400310 wikitext text/x-wiki rygr7jsb5772ovzc13clb1ivbp2cq1u 2400311 2400310 2026-04-09T18:52:56Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with sync 2400311 wikitext text/x-wiki b1wmhcbi2hima3lrt57yg1uup3oyho2 2400312 2400311 2026-04-09T18:56:44Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269572|Revert^5 "Use envoy for swift inside mediawiki"]] (duration: 05m 53s) 2400312 wikitext text/x-wiki szgv9kylzowzaors5ya4ijl95wt7m4u 2400313 2400312 2026-04-09T18:58:07Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400313 wikitext text/x-wiki 4mkgdqoesy3xp7amf7z21es8fbunhyi 2400314 2400313 2026-04-09T18:58:12Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400314 wikitext text/x-wiki 0k3jncv9egq39n48439hh6n4hjy2wwc 2400316 2400314 2026-04-09T19:02:05Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400316 wikitext text/x-wiki he4mtzllpgfiuczr2k34e0wwbqmqsyc 2400317 2400316 2026-04-09T19:02:10Z Stashbot 7414 otto@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400317 wikitext text/x-wiki 5z6w09bke1ntlbw3bm0vsiie8geanji 2400318 2400317 2026-04-09T19:04:30Z Stashbot 7414 inflatador: bking@apt1002 delete old haproxy pkgs P90343 2400318 wikitext text/x-wiki g0ghyt1c8thlatajmf45sa9l32qsq6v 2400319 2400318 2026-04-09T19:06:06Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.dns.netbox 2400319 wikitext text/x-wiki rdjxk622s8wtzvqd52jfpa2may19jks 2400320 2400319 2026-04-09T19:09:08Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2400320 wikitext text/x-wiki lh9asujfb1hqooz7kae0cnotqt0988l 2400321 2400320 2026-04-09T19:09:35Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.network.configure-switch-interfaces for host ganeti1055 2400321 wikitext text/x-wiki 7f6jleee0r19jp66y4pa6vitqyf5zxi 2400322 2400321 2026-04-09T19:10:40Z Stashbot 7414 vriley@cumin1003: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ganeti1055 2400322 wikitext text/x-wiki bl2nyjww9sxdfktkt6qsta7dwvqnjfi 2400323 2400322 2026-04-09T19:11:23Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host ganeti1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400323 wikitext text/x-wiki 7xzjtxh08h1ucdo2rwdkm2jn1c90cho 2400325 2400323 2026-04-09T19:15:38Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400325 wikitext text/x-wiki ac4zll360ohlv0z8727zbei0pjtzi4w 2400326 2400325 2026-04-09T19:16:33Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host ganeti1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400326 wikitext text/x-wiki 1db4whozvmy1trs9cj5xbkh0gc1g782 2400327 2400326 2026-04-09T19:16:50Z Stashbot 7414 inflatador: bking@apt1002 sudo -E reprepro -C thirdparty/opensearch2 copy trixie-wikimedia bookworm-wikimedia opensearch T422860 2400327 wikitext text/x-wiki 6wqctt81dtjhk105b7p2wuealsa8a5f 2400328 2400327 2026-04-09T19:18:01Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400328 wikitext text/x-wiki giag5ceiex0jla9frzl0yde7z6px4h1 2400329 2400328 2026-04-09T19:20:03Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host ganeti1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400329 wikitext text/x-wiki 2izcl2kq6oh2yqkjn7xme2n3g5jdqfv 2400330 2400329 2026-04-09T19:24:48Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400330 wikitext text/x-wiki hlcv74geyc58qrng7e3be32ke1ia6z3 2400331 2400330 2026-04-09T19:29:14Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host ganeti1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400331 wikitext text/x-wiki dblo5cfr7cq3grwjn6j5xgckwszegd8 2400332 2400331 2026-04-09T19:30:23Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400332 wikitext text/x-wiki kmvp5bm71718slfdrwmjp5two5aauog 2400333 2400332 2026-04-09T19:44:53Z Stashbot 7414 vriley@cumin1003: START - Cookbook sre.hosts.provision for host ganeti1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400333 wikitext text/x-wiki d2eylgtjd9akx0g6i7ib3hq557gdlh3 2400334 2400333 2026-04-09T19:46:13Z Stashbot 7414 vriley@cumin1003: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400334 wikitext text/x-wiki dcz2276h2tkiwts3nanq1j3kemlek2u 2400337 2400334 2026-04-09T20:00:53Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.o11y.thanos-compact-restart cookbook test (patch id: 1260650) 2400337 wikitext text/x-wiki trw3mb3hm5dgyxpfmdf1pf0qazqmoer 2400338 2400337 2026-04-09T20:03:14Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.o11y.thanos-compact-restart (exit_code=0) cookbook test (patch id: 1260650) 2400338 wikitext text/x-wiki 0aw8a5rllst9l0gyfiexspc52073ty2 2400340 2400338 2026-04-09T20:04:24Z Stashbot 7414 andrew@cumin2002: START - Cookbook sre.hosts.reboot-single for host cloudcephmon2006-dev.codfw.wmnet 2400340 wikitext text/x-wiki s01hi46eoktxbeotkdd5r5ah3e3x94z 2400341 2400340 2026-04-09T20:07:58Z Stashbot 7414 aude@deploy1003: Started scap sync-world: Backport for [[gerrit:1269475|Opt-in new accounts to ReadingLists beta feature on testwiki (T422833)]], [[gerrit:1047441|Add new protection level (edituserprotected) for nowiki (T367943)]], [[gerrit:1268680|Turn on Parsoid Read Views for dewiki (T422524)]] 2400341 wikitext text/x-wiki eviio4ylx5arp8p7ak3og8sc9rz64a3 2400342 2400341 2026-04-09T20:09:40Z Stashbot 7414 aude@deploy1003: cscott, jhsoby, aude: Backport for [[gerrit:1269475|Opt-in new accounts to ReadingLists beta feature on testwiki (T422833)]], [[gerrit:1047441|Add new protection level (edituserprotected) for nowiki (T367943)]], [[gerrit:1268680|Turn on Parsoid Read Views for dewiki (T422524)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400342 wikitext text/x-wiki jeuoxaw90yd556p3euu2rzwlndsuhim 2400343 2400342 2026-04-09T20:10:44Z Stashbot 7414 andrew@cumin2002: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcephmon2006-dev.codfw.wmnet 2400343 wikitext text/x-wiki nvd13u118hws2mslogll9ofrvrfzcbr 2400344 2400343 2026-04-09T20:13:05Z Stashbot 7414 aude@deploy1003: cscott, jhsoby, aude: Continuing with sync 2400344 wikitext text/x-wiki 3m8kkzq2s36qrgzbk9onvnmh2q3o3oy 2400345 2400344 2026-04-09T20:15:36Z Stashbot 7414 tappof@cumin1003: START - Cookbook sre.o11y.thanos-compact-restart rebalance blocks across compactor instances (patch id: 1265429) 2400345 wikitext text/x-wiki bcs1vt7hr6snji7ushhpugnfn9m51f8 2400346 2400345 2026-04-09T20:17:02Z Stashbot 7414 aude@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269475|Opt-in new accounts to ReadingLists beta feature on testwiki (T422833)]], [[gerrit:1047441|Add new protection level (edituserprotected) for nowiki (T367943)]], [[gerrit:1268680|Turn on Parsoid Read Views for dewiki (T422524)]] (duration: 09m 04s) 2400346 wikitext text/x-wiki seq4lcqgh84y0t1atm1h52qcnblmbny 2400347 2400346 2026-04-09T20:17:48Z Stashbot 7414 tappof@cumin1003: END (PASS) - Cookbook sre.o11y.thanos-compact-restart (exit_code=0) rebalance blocks across compactor instances (patch id: 1265429) 2400347 wikitext text/x-wiki qe3s97qbzf43hm94n6z67m6emhqcajo 2400348 2400347 2026-04-09T20:18:53Z Stashbot 7414 aude@deploy1003: Started scap sync-world: Backport for [[gerrit:1269552|Make onboarding dialog a little less eager beaver 🦫 (T421942)]] 2400348 wikitext text/x-wiki t1dn1bp26nvraxbotodzsr8aiasadhg 2400349 2400348 2026-04-09T20:20:34Z Stashbot 7414 aude@deploy1003: aude: Backport for [[gerrit:1269552|Make onboarding dialog a little less eager beaver 🦫 (T421942)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400349 wikitext text/x-wiki gjmwujn8231avppojq03cr1cenz5zij 2400350 2400349 2026-04-09T20:25:07Z Stashbot 7414 aude@deploy1003: aude: Continuing with sync 2400350 wikitext text/x-wiki 5cosfggpdmninkqo3qn61nzqnrnvv14 2400351 2400350 2026-04-09T20:27:16Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.dns.netbox 2400351 wikitext text/x-wiki rlp8g7wwxqlddq96w8aw7bdglx35i30 2400352 2400351 2026-04-09T20:28:58Z Stashbot 7414 aude@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269552|Make onboarding dialog a little less eager beaver 🦫 (T421942)]] (duration: 10m 05s) 2400352 wikitext text/x-wiki l0zb4idwbxf6swf6tkke99pd4nckfic 2400353 2400352 2026-04-09T20:31:42Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding apus-be2005-6 and phab2003 to codfw - jhancock@cumin2002" 2400353 wikitext text/x-wiki 5bdvqccxrojubqepbxqbubz6rnu05vh 2400354 2400353 2026-04-09T20:31:47Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding apus-be2005-6 and phab2003 to codfw - jhancock@cumin2002" 2400354 wikitext text/x-wiki mg049zwpsvsopi8ugsdt0zhba5he7mp 2400355 2400354 2026-04-09T20:31:48Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2400355 wikitext text/x-wiki hxoffpl56go8uedno326ggbq4bymhkf 2400356 2400355 2026-04-09T20:31:58Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host apus-be2005 2400356 wikitext text/x-wiki i9y9y1e9g5glgalni5e955tvl0yg0xt 2400357 2400356 2026-04-09T20:37:50Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply 2400357 wikitext text/x-wiki 3f3qep6yr811rpfj5ur9047zy8onrsi 2400358 2400357 2026-04-09T20:39:35Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply 2400358 wikitext text/x-wiki 3cm5yyqdkxsl97yttxkfmsor5bi8m1m 2400360 2400358 2026-04-09T20:45:05Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host apus-be2005 2400360 wikitext text/x-wiki ap54gzkrtvrorjxs32c700bp726oma4 2400361 2400360 2026-04-09T20:45:13Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host apus-be2006 2400361 wikitext text/x-wiki ozgjgfukldrazc6g8i66kb3wz59yxir 2400362 2400361 2026-04-09T20:45:25Z Stashbot 7414 inflatador: reprepro --noskipold --component thirdparty/opensearch2 update trixie-wikimedia T422860 2400362 wikitext text/x-wiki 4ddy1mnbjkr7krlcsrt2en4vcndwm1p 2400363 2400362 2026-04-09T20:50:16Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host apus-be2006 2400363 wikitext text/x-wiki acm0951wbt10t59xkaui6q0saofa5jh 2400364 2400363 2026-04-09T20:50:25Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.network.configure-switch-interfaces for host phab2003 2400364 wikitext text/x-wiki lor4d1dbdzj1ny30iuva0reqvfok88e 2400365 2400364 2026-04-09T20:50:36Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host phab2003 2400365 wikitext text/x-wiki c1gt75ioz1g3kckky5wfy4n02va89ri 2400366 2400365 2026-04-09T20:56:27Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host phab2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400366 wikitext text/x-wiki 868axekr3u9itylmk1bvunli97p68m4 2400367 2400366 2026-04-09T21:06:24Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host phab2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400367 wikitext text/x-wiki 4sgncmd5xh1okytgxelkdm8mgh4znxx 2400369 2400367 2026-04-09T21:11:17Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host phab2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400369 wikitext text/x-wiki c4gvfyp3gvdkh7th3jb08g2qiuqknv6 2400371 2400369 2026-04-09T21:19:05Z Stashbot 7414 cscott@deploy1003: Started scap sync-world: Backport for [[gerrit:1269653|ParsoidLanguageConverter: Don't convert inside <style> elements (T422879)]] 2400371 wikitext text/x-wiki jiqjorm0shgs7u5hvoz5o4rxvn7tuzu 2400372 2400371 2026-04-09T21:20:46Z Stashbot 7414 cscott@deploy1003: cscott: Backport for [[gerrit:1269653|ParsoidLanguageConverter: Don't convert inside <style> elements (T422879)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400372 wikitext text/x-wiki haaqmotjcjftc9caxno74xfu14asvtl 2400373 2400372 2026-04-09T21:21:19Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen: apply 2400373 wikitext text/x-wiki los59rn8acqyrxpzvcequ9an0jgrmfe 2400374 2400373 2026-04-09T21:21:41Z Stashbot 7414 sfaci@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen: apply 2400374 wikitext text/x-wiki f2ljxbe27oelivrwovjzat31zsqa68t 2400375 2400374 2026-04-09T21:22:10Z Stashbot 7414 cscott@deploy1003: cscott: Continuing with sync 2400375 wikitext text/x-wiki b6t561a71or8fupxelx51t8achnh68u 2400376 2400375 2026-04-09T21:25:17Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host phab2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400376 wikitext text/x-wiki d5poisc3v9wyaezk6rtr63e7or4hhqn 2400377 2400376 2026-04-09T21:25:58Z Stashbot 7414 cscott@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269653|ParsoidLanguageConverter: Don't convert inside <style> elements (T422879)]] (duration: 06m 52s) 2400377 wikitext text/x-wiki 25xd9hi816un1i4qkc5sackljaherdf 2400378 2400377 2026-04-09T21:27:59Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host phab2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400378 wikitext text/x-wiki ck8v9kl3jp3j1tub4zenita1qlqxmcz 2400380 2400378 2026-04-09T21:57:32Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host phab2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400380 wikitext text/x-wiki ft4jgqorrnd2ee8shdxw6r17bd2906o 2400382 2400380 2026-04-09T22:58:21Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host apus-be2005.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400382 wikitext text/x-wiki af8r0zk5jkq4hemlh1hwm293t3iunmc 2400383 2400382 2026-04-09T22:58:52Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.provision for host apus-be2006.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400383 wikitext text/x-wiki d2f82lpf5qrqxwsz432j2qe9rjzxikg 2400384 2400383 2026-04-09T23:11:54Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host apus-be2006.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400384 wikitext text/x-wiki 8yfdkcj8x0bbu6wvo1vxaa11azf2m7i 2400385 2400384 2026-04-09T23:13:52Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host apus-be2005.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED 2400385 wikitext text/x-wiki jp7tpsakbm4no88p5gmc8zajpt19v63 2400386 2400385 2026-04-09T23:14:20Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['apus-be2005'] 2400386 wikitext text/x-wiki 1379j8o73tav0tzxau5u7wlppd228pc 2400387 2400386 2026-04-09T23:14:33Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['apus-be2005'] 2400387 wikitext text/x-wiki tepnha5b2kchoup5oem4zf8096o4vr5 2400388 2400387 2026-04-09T23:15:19Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host apus-be2005.codfw.wmnet with OS bookworm 2400388 wikitext text/x-wiki nsmgmlamaysk5zlv347ap8f9p3sb2za 2400389 2400388 2026-04-09T23:15:41Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host apus-be2006.codfw.wmnet with OS bookworm 2400389 wikitext text/x-wiki tnxlne9p9s9d2ggifo7umy0bhj6pltj 2400391 2400389 2026-04-09T23:43:38Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on apus-be2006.codfw.wmnet with reason: host reimage 2400391 wikitext text/x-wiki d0backgmjfgxvqce3s3vfwj8ati6ebj 2400392 2400391 2026-04-09T23:48:27Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on apus-be2006.codfw.wmnet with reason: host reimage 2400392 wikitext text/x-wiki 48idkma4sxbrukdhccvc6hffkybo4zl 2400393 2400392 2026-04-09T23:58:03Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.downtime for 2:00:00 on apus-be2005.codfw.wmnet with reason: host reimage 2400393 wikitext text/x-wiki p361glm574ps3o9rrx4mur4t6uor491 2400395 2400393 2026-04-10T00:00:43Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1269720|Disable query pages on testcommonswiki not compatible with split (T421914)]] 2400395 wikitext text/x-wiki hla1uh1dljrwbpuq6z7f7md9p3vbv0r 2400396 2400395 2026-04-10T00:02:27Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1269720|Disable query pages on testcommonswiki not compatible with split (T421914)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400396 wikitext text/x-wiki 8nnaq368i5a1f0xxcrpbjhvd609v0zd 2400397 2400396 2026-04-10T00:03:50Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on apus-be2005.codfw.wmnet with reason: host reimage 2400397 wikitext text/x-wiki 5ifj7mh2fia8axj1z8p71mu9uzl6p4d 2400398 2400397 2026-04-10T00:04:13Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with sync 2400398 wikitext text/x-wiki 9wyz4tq0xh59nrj0q7tslxkh36qv67d 2400399 2400398 2026-04-10T00:07:06Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" 2400399 wikitext text/x-wiki mtpnfvuh2b274z9d6n2mbpasx2gxcfw 2400400 2400399 2026-04-10T00:07:44Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" 2400400 wikitext text/x-wiki 9xox20ko3rxdbl2ywfa82jdn8b2y8lg 2400401 2400400 2026-04-10T00:07:45Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host apus-be2006.codfw.wmnet with OS bookworm 2400401 wikitext text/x-wiki czyb23s7b8sn4kyxo9yf178rfgz94yu 2400402 2400401 2026-04-10T00:08:00Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269720|Disable query pages on testcommonswiki not compatible with split (T421914)]] (duration: 07m 17s) 2400402 wikitext text/x-wiki kvjtv5yoxgd8e1u6tmn72skh00supfe 2400403 2400402 2026-04-10T00:21:19Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" 2400403 wikitext text/x-wiki slvqo6pkjfzlia8gyaz054lzled3lc6 2400404 2400403 2026-04-10T00:26:09Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jhancock@cumin2002" 2400404 wikitext text/x-wiki o0j650ty6h5rg8hy4fwfctgd6k1acvz 2400405 2400404 2026-04-10T00:26:11Z Stashbot 7414 jhancock@cumin2002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host apus-be2005.codfw.wmnet with OS bookworm 2400405 wikitext text/x-wiki jlasx1tnny4ni25tlw9mg2nlicentwm 2400406 2400405 2026-04-10T00:29:29Z Stashbot 7414 zabe: marked 425 content rows as bad # T393237 2400406 wikitext text/x-wiki rli45h71la1l0l6a7k28ywazaebzj1j 2400407 2400406 2026-04-10T00:40:40Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1269086|Start reading from new file tables on enwiki (T416548)]] 2400407 wikitext text/x-wiki nhhrs1604zcohbw2sz752zcqyeqlong 2400408 2400407 2026-04-10T00:42:22Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1269086|Start reading from new file tables on enwiki (T416548)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400408 wikitext text/x-wiki 9jgemii5k6yyg3ixq80691jv0ciqjpz 2400409 2400408 2026-04-10T00:43:05Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with sync 2400409 wikitext text/x-wiki 8ugm9ua4iuq5m5fp49u0stj3a0izlwz 2400410 2400409 2026-04-10T00:46:52Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269086|Start reading from new file tables on enwiki (T416548)]] (duration: 06m 11s) 2400410 wikitext text/x-wiki cafm69w8wmqmwv6lrnd5y148ts9w3su 2400411 2400410 2026-04-10T00:48:35Z Stashbot 7414 zabe@deploy1003: Started scap sync-world: Backport for [[gerrit:1269744|Stop setting specific virtual domain for link tables (T421914)]] 2400411 wikitext text/x-wiki 8aeirinlxy04qp2yijw4hjljg6zan80 2400412 2400411 2026-04-10T00:50:22Z Stashbot 7414 zabe@deploy1003: zabe: Backport for [[gerrit:1269744|Stop setting specific virtual domain for link tables (T421914)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2400412 wikitext text/x-wiki 7ysfhr8wa2h1ixgc46z23f4h78qtjr3 2400413 2400412 2026-04-10T00:50:40Z Stashbot 7414 zabe@deploy1003: zabe: Continuing with sync 2400413 wikitext text/x-wiki qbb4ylelrwmbfg01ns0zgydt0u6wr37 2400414 2400413 2026-04-10T00:54:26Z Stashbot 7414 zabe@deploy1003: Finished scap sync-world: Backport for [[gerrit:1269744|Stop setting specific virtual domain for link tables (T421914)]] (duration: 05m 51s) 2400414 wikitext text/x-wiki slkof2dqe37ailoxedbwdwflxknlkdu 2400415 2400414 2026-04-10T00:57:47Z Stashbot 7414 rzl@deploy1003: helmfile [staging] START helmfile.d/services/wikifunctions: apply 2400415 wikitext text/x-wiki so8yxdchzpgodlidq15yu9b13mta9nt 2400416 2400415 2026-04-10T00:57:56Z Stashbot 7414 rzl@deploy1003: helmfile [staging] DONE helmfile.d/services/wikifunctions: apply 2400416 wikitext text/x-wiki lukytjp509mq9dxcb6f97p0adulgczz 2400418 2400416 2026-04-10T01:23:00Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] START helmfile.d/services/wikifunctions: apply 2400418 wikitext text/x-wiki 3c5f1y948gk2p859rng8vfc12lou6i5 2400419 2400418 2026-04-10T01:23:56Z Stashbot 7414 rzl@deploy1003: helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply 2400419 wikitext text/x-wiki a3cbk8dem4u4ry2ng31ynaeqxl6quwi 2400420 2400419 2026-04-10T01:25:43Z Stashbot 7414 rzl@deploy1003: helmfile [codfw] START helmfile.d/services/wikifunctions: apply 2400420 wikitext text/x-wiki pyr4l89defw65tmfv6egtg90en57wcq 2400421 2400420 2026-04-10T01:26:37Z Stashbot 7414 rzl@deploy1003: helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply 2400421 wikitext text/x-wiki epvwx3o60zz3r24bxwk2rdcy67rbcao 2400423 2400421 2026-04-10T05:05:46Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.reimage for host clouddb1019.eqiad.wmnet with OS trixie 2400423 wikitext text/x-wiki jwmldavu4omg1qmy0ymdyoz8lmkn9rg 2400424 2400423 2026-04-10T06:26:52Z Stashbot 7414 marostegui@cumin1003: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddb1019.eqiad.wmnet with OS trixie 2400424 wikitext text/x-wiki f50g66ymiv8me9c4ah5p738cwi5gbl6 2400425 2400424 2026-04-10T07:09:07Z Stashbot 7414 aokoth@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on gitlab1003.wikimedia.org with reason: Upgrade 2400425 wikitext text/x-wiki r71f1x3d2hwst2qoufj0r554g5mziss 2400426 2400425 2026-04-10T07:29:22Z Stashbot 7414 jelto@dns1004: START - running authdns-update 2400426 wikitext text/x-wiki 6a18gytcz37ssl9wi4tnh035wj09fho 2400427 2400426 2026-04-10T07:30:48Z Stashbot 7414 jelto@dns1004: END - running authdns-update 2400427 wikitext text/x-wiki p3islsgb743dm9nn0ez42adaxpswlm8 2400428 2400427 2026-04-10T08:19:26Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . 2400428 wikitext text/x-wiki 0yw18ox698lqnpffgmrdhxohikauxyd 2400437 2400428 2026-04-10T08:42:45Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400437 wikitext text/x-wiki f9gnr2a4kztz7trtxksz2o3rel87gd8 2400438 2400437 2026-04-10T08:44:32Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400438 wikitext text/x-wiki d0pcc8szx3hd58l6isguo5b9lwmq4xl 2400439 2400438 2026-04-10T08:44:57Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400439 wikitext text/x-wiki k4boo7pk7eiw2qz7rt6mthuik8jd1do 2400440 2400439 2026-04-10T08:45:00Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400440 wikitext text/x-wiki 7uq7knwcesi4yxdhojrudyn51xus5e4 2400441 2400440 2026-04-10T08:45:14Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400441 wikitext text/x-wiki i3w6pslcxyl4jcrghk1100epgcp8fcg 2400442 2400441 2026-04-10T08:45:17Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400442 wikitext text/x-wiki 4gf3f38xh66zy3i5h85ysxascmbhy1l 2400443 2400442 2026-04-10T08:54:15Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400443 wikitext text/x-wiki 7bpbttugwbroz4xplny0futqs02itj7 2400444 2400443 2026-04-10T08:54:19Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400444 wikitext text/x-wiki s7sxbe0ez3z9e2tknsagi73ebw7mrvl 2400445 2400444 2026-04-10T08:54:34Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400445 wikitext text/x-wiki nk1qgockfczh53tmp3c5zibzzuzv7ni 2400446 2400445 2026-04-10T08:54:41Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400446 wikitext text/x-wiki hrzjbpiat2q21hh1y762vw6yc8asmq3 2400447 2400446 2026-04-10T08:55:00Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . 2400447 wikitext text/x-wiki iva7rfwaula75zon95yr8ta25nxcgmu 2400448 2400447 2026-04-10T09:00:25Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400448 wikitext text/x-wiki s78jkplokupvnat6ssc7o32wufx2bk0 2400449 2400448 2026-04-10T09:02:22Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400449 wikitext text/x-wiki ic7iv9ikdlb0v0yhjfmowo0jbj6hyoc 2400450 2400449 2026-04-10T09:02:59Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400450 wikitext text/x-wiki q72p3aznoq43hzhhl6in590w2t9cmyf 2400451 2400450 2026-04-10T09:04:46Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400451 wikitext text/x-wiki lb10eeerb316u25jkv8wt7eypq4fhj9 2400452 2400451 2026-04-10T09:13:03Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400452 wikitext text/x-wiki hsh3c3g4y0681iwgv65uphzk73unhy1 2400453 2400452 2026-04-10T09:14:50Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400453 wikitext text/x-wiki 7oteygqwrkth70u5r2q6sgp0mi2t3pn 2400454 2400453 2026-04-10T09:15:22Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400454 wikitext text/x-wiki 7smc06jfj9eby9x5osv0c61zkgzuwcy 2400455 2400454 2026-04-10T09:16:26Z Stashbot 7414 fceratto@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2147.codfw.wmnet with reason: Maintenance 2400455 wikitext text/x-wiki 2v0m1lsfg4isvl4hf6p3na32golis0p 2400456 2400455 2026-04-10T09:17:13Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400456 wikitext text/x-wiki q1gxkjsxnyg532qka21ue5nhhjfo86q 2400457 2400456 2026-04-10T09:17:16Z Stashbot 7414 fceratto@cumin2002: dbctl commit (dc=all): 'Depooling db2147 (T419635)', diff saved to https://phabricator.wikimedia.org/P90346 and previous config saved to /var/cache/conftool/dbconfig/20260410-091713-fceratto.json 2400457 wikitext text/x-wiki dcl6nucnbhzkhk7r0bhmggblsoz94p3 2400458 2400457 2026-04-10T09:18:35Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . 2400458 wikitext text/x-wiki iftgscyyzqxasq4c16tzzyqplipmtgm 2400460 2400458 2026-04-10T09:21:14Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] START helmfile.d/services/rest-gateway: apply 2400460 wikitext text/x-wiki hlxbpzjixm1l98r54fp15ezmdcdruwu 2400461 2400460 2026-04-10T09:21:23Z Stashbot 7414 cgoubert@deploy1003: helmfile [staging] DONE helmfile.d/services/rest-gateway: apply 2400461 wikitext text/x-wiki 4mnahogz7j0kc0i91zbof2z1zgdical 2400462 2400461 2026-04-10T09:21:32Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] START helmfile.d/services/rest-gateway: apply 2400462 wikitext text/x-wiki q1so2cbec2tsazpcscf2qypyvd22yby 2400463 2400462 2026-04-10T09:21:48Z Stashbot 7414 cgoubert@deploy1003: helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply 2400463 wikitext text/x-wiki awqleesn51msffokjnfyoqvsesdx2hv 2400464 2400463 2026-04-10T09:22:06Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] START helmfile.d/services/rest-gateway: apply 2400464 wikitext text/x-wiki 58thrym0sww70lu5pgs9x7xggd461ok 2400465 2400464 2026-04-10T09:22:22Z Stashbot 7414 cgoubert@deploy1003: helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply 2400465 wikitext text/x-wiki 4zfiw2xh4kyuqfrxqwyehavt6dx8brz 2400466 2400465 2026-04-10T09:24:22Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/jaeger: sync 2400466 wikitext text/x-wiki 7g6zblqtbaogprt24v25ml3z5rxph0i 2400467 2400466 2026-04-10T09:24:27Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/jaeger: sync 2400467 wikitext text/x-wiki mrqo5332shz2bdj3bjjbwb3gergyjqn 2400468 2400467 2026-04-10T09:24:29Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply 2400468 wikitext text/x-wiki 3v1pt0sv1yn93058ggam0vpnvd3m995 2400469 2400468 2026-04-10T09:24:38Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-codfw] START helmfile.d/aux-k8s-services/jaeger: sync 2400469 wikitext text/x-wiki aa5g1661vmb39cxdqkooqobpo3lhvpr 2400470 2400469 2026-04-10T09:24:46Z Stashbot 7414 elukey@deploy1003: helmfile [aux-k8s-codfw] DONE helmfile.d/aux-k8s-services/jaeger: sync 2400470 wikitext text/x-wiki c312gs88i494ws6elvr1fewb9wtl5iv 2400471 2400470 2026-04-10T09:24:48Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply 2400471 wikitext text/x-wiki 5k8l6cl7vezvafb4x9aeobeyaf7t0vc 2400472 2400471 2026-04-10T09:30:12Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400472 wikitext text/x-wiki gm1vw45liagbwdvoijtusmpluohe7ic 2400473 2400472 2026-04-10T09:31:32Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400473 wikitext text/x-wiki s7aztqi1h90vdo6yalcr51cwn1spsi8 2400474 2400473 2026-04-10T09:35:56Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400474 wikitext text/x-wiki p2a49umz1xqeff6jwcy71sanibb0lhb 2400475 2400474 2026-04-10T09:36:03Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400475 wikitext text/x-wiki suz51w7cyc52b5iulu80gqjjbuehyxl 2400476 2400475 2026-04-10T09:39:50Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . 2400476 wikitext text/x-wiki 9t4wajj5e8q1xj66sqcqm46mjcjvh21 2400478 2400476 2026-04-10T09:48:06Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400478 wikitext text/x-wiki azjs5ledb82rijae7s19fowt37ask03 2400481 2400478 2026-04-10T09:48:20Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400481 wikitext text/x-wiki c3iy32aku7sw5z942p1orp1wsa6x6cl 2400483 2400481 2026-04-10T09:50:07Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . 2400483 wikitext text/x-wiki 7l19z2gb09oi8gpgkrxruws2soi9uc9 2400487 2400483 2026-04-10T09:57:54Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400487 wikitext text/x-wiki 4xuzt5cidndsl5f9egi4x6nsgiwgx8e 2400489 2400487 2026-04-10T09:58:07Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400489 wikitext text/x-wiki q6zt9yc4k7wljpbuc2shlqbujvrg82x 2400491 2400489 2026-04-10T10:03:29Z Stashbot 7414 aokoth@cumin1003: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1004.wikimedia.org with reason: Security Release - T422668 2400491 wikitext text/x-wiki lrzby0wv5fpdgtbswes3tw28gszs524 2400492 2400491 2026-04-10T10:07:52Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400492 wikitext text/x-wiki qg0oxdtzltib14uwuixn10ngq9z0s49 2400493 2400492 2026-04-10T10:09:40Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400493 wikitext text/x-wiki j13id8yq2utk6zc2mxqybceormkj9j9 2400494 2400493 2026-04-10T10:09:48Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400494 wikitext text/x-wiki 05qukk7g5l9q7k4f3wslenhr840dapd 2400495 2400494 2026-04-10T10:10:08Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400495 wikitext text/x-wiki m8ey623gw5v8ma2t03di0ze6xhb30jg 2400496 2400495 2026-04-10T10:10:23Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400496 wikitext text/x-wiki 5ycmgl3gyalbda2glg9h5fjqq9noabl 2400497 2400496 2026-04-10T10:11:07Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400497 wikitext text/x-wiki dui0ie3vyiwfglihwv8rv8lt0q63034 2400498 2400497 2026-04-10T10:11:17Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400498 wikitext text/x-wiki cmfz3xqnyxaa7it67hf3g0xvnv8e5ko 2400499 2400498 2026-04-10T10:11:37Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400499 wikitext text/x-wiki h3bv7xwa9xql43j6ssqetwss0v0c5nv 2400501 2400499 2026-04-10T10:14:29Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400501 wikitext text/x-wiki jt377d13losql7igrimm9xdp26e1sp6 2400502 2400501 2026-04-10T10:16:18Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400502 wikitext text/x-wiki 9l8eua563etdzgnvsatbhvulqpud2j6 2400503 2400502 2026-04-10T10:16:25Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400503 wikitext text/x-wiki 9rjylnlginq3xrit14bv1nyt1bd65hr 2400504 2400503 2026-04-10T10:18:15Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400504 wikitext text/x-wiki n38iaj7wazww97wfx1vbsr129lqovd9 2400505 2400504 2026-04-10T10:19:08Z Stashbot 7414 vgutierrez: upload haproxy 2.8.20 to thirdparty/haproxy28 for bookworm-wikimedia (apt.wm.o) - T422926 2400505 wikitext text/x-wiki 80jw5r0mludlsw6vbpaoju74l691ntk 2400506 2400505 2026-04-10T10:20:15Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400506 wikitext text/x-wiki eoajwpcsysork38g7mff8besej9hrek 2400507 2400506 2026-04-10T10:20:22Z Stashbot 7414 javiermonton@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich-next: apply 2400507 wikitext text/x-wiki 4dc6k1lbtudhu0j0lww8df5762hcjnb 2400508 2400507 2026-04-10T10:21:14Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . 2400508 wikitext text/x-wiki 26dngi3zeb4gnthy00ip2fjvhjur7b6 2400512 2400508 2026-04-10T10:25:44Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400512 wikitext text/x-wiki 2fv5k0tn23bo7m8xqpvt3177tajdmzz 2400513 2400512 2026-04-10T10:27:32Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400513 wikitext text/x-wiki 94z87gsazo65cwhre40mez8boewm3zj 2400514 2400513 2026-04-10T10:27:39Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400514 wikitext text/x-wiki rrl2px9fblui4j4qr1enkpprpovsqik 2400515 2400514 2026-04-10T10:30:05Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400515 wikitext text/x-wiki l4gci40ca20vdcx3s3vlczaf2xcl2uz 2400516 2400515 2026-04-10T10:58:12Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400516 wikitext text/x-wiki sjdnygsywvhqb4hxoiy9m5pgr1y455k 2400517 2400516 2026-04-10T11:00:02Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400517 wikitext text/x-wiki c592ru8tgz5aso38ujgwhsiawifsjrs 2400518 2400517 2026-04-10T11:00:38Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400518 wikitext text/x-wiki pu7v1g9k418ukttflf5b6oskj93wrm5 2400519 2400518 2026-04-10T11:02:30Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400519 wikitext text/x-wiki 2p6o8wbbctcae55w0tccxuvkygkvolf 2400520 2400519 2026-04-10T11:02:44Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . 2400520 wikitext text/x-wiki lttrrzcj6b4p33moo9eh5eq7znfhpbv 2400521 2400520 2026-04-10T11:04:41Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . 2400521 wikitext text/x-wiki ch0asjns1wny56iadjw99awb5ourre6 2400522 2400521 2026-04-10T11:04:59Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . 2400522 wikitext text/x-wiki irv66e83x53gj5368e03ycjo18yizay 2400523 2400522 2026-04-10T11:10:47Z Stashbot 7414 aokoth@cumin1003: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1004.wikimedia.org with reason: Security Release - T422668 2400523 wikitext text/x-wiki fsaq25vmtjztw3xw6ggcyxpkaik0dup 2400525 2400523 2026-04-10T11:15:05Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400525 wikitext text/x-wiki 5yqki2b5pgtusm0lcd4v80mra4v2nxt 2400526 2400525 2026-04-10T11:16:53Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400526 wikitext text/x-wiki suagepuqc3f2u4292h21is9l8qzjon8 2400527 2400526 2026-04-10T11:16:57Z Stashbot 7414 fceratto@cumin2002: dbctl commit (dc=all): 'Repooling after maintenance db2147 (T419635)', diff saved to https://phabricator.wikimedia.org/P90347 and previous config saved to /var/cache/conftool/dbconfig/20260410-111654-fceratto.json 2400527 wikitext text/x-wiki 8ah5y57h352jdn7prcomqsak33f7bx0 2400528 2400527 2026-04-10T11:19:09Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. 2400528 wikitext text/x-wiki 12btcghxbtw60xpj61mffmku7s8avp8 2400529 2400528 2026-04-10T11:20:59Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. 2400529 wikitext text/x-wiki exo2bbp5l5wanjue7knbsyhr6vczzwz 2400530 2400529 2026-04-10T11:22:54Z Stashbot 7414 dpogorzelski@deploy1003: helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'edit-check' for release 'main' . 2400530 wikitext text/x-wiki sef5soem93ge1iqason9mqobc9jkiat 2400531 2400530 2026-04-10T11:27:44Z Stashbot 7414 fceratto@cumin2002: dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P90348 and previous config saved to /var/cache/conftool/dbconfig/20260410-112742-fceratto.json 2400531 wikitext text/x-wiki 96rheyyve8ac788l02fub2trkyx3huj 2400533 2400531 2026-04-10T11:38:33Z Stashbot 7414 fceratto@cumin2002: dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P90349 and previous config saved to /var/cache/conftool/dbconfig/20260410-113830-fceratto.json 2400533 wikitext text/x-wiki jt1inla9c3ha46attwwgwbsth7iep7t 2400537 2400533 2026-04-10T11:49:24Z Stashbot 7414 fceratto@cumin2002: dbctl commit (dc=all): 'Repooling after maintenance db2147 (T419635)', diff saved to https://phabricator.wikimedia.org/P90350 and previous config saved to /var/cache/conftool/dbconfig/20260410-114919-fceratto.json 2400537 wikitext text/x-wiki 52clm917q5krdp2b8iqadznfjysaaev 2400538 2400537 2026-04-10T11:49:28Z Stashbot 7414 fceratto@cumin2002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2155.codfw.wmnet with reason: Maintenance 2400538 wikitext text/x-wiki ls98a2rhxmurhpti4505xpxmzdlwu2j 2400539 2400538 2026-04-10T11:50:19Z Stashbot 7414 fceratto@cumin2002: dbctl commit (dc=all): 'Depooling db2155 (T419635)', diff saved to https://phabricator.wikimedia.org/P90351 and previous config saved to /var/cache/conftool/dbconfig/20260410-115015-fceratto.json 2400539 wikitext text/x-wiki pxwjat7q4cupqn94zn7usxtoe3d0s4e 0 17290 2400188 2399972 2026-04-09T14:23:20Z Stashbot 7414 hashar: Tagged Quibble 1.17.0 @ 864381c6b63bdbcd8c74a3162c406fffcaaf8694 2400188 wikitext text/x-wiki s8vh3zg6093ik3xdgsdb9fwiip949cl 2400211 2400188 2026-04-09T14:58:29Z Stashbot 7414 hashar: Upgrading Quibble jobs to 1.17.0 2400211 wikitext text/x-wiki 2xs4nxsiuhkt9tm4mnodspmlg6wzbo9 2400214 2400211 2026-04-09T15:13:07Z Stashbot 7414 hashar: Rolling back Quibble jobs to 1.16.0 (api-testing stage fails due to missing npm install step` 2400214 wikitext text/x-wiki 2d67yxb7b12q90q7n3nito0mhtjxym6 2400485 2400214 2026-04-10T09:51:08Z Stashbot 7414 hashar: Tag Quibble 1.17.1 @ 0a1ab3b7c3dfee36c9bc2e9b049957d94e190e85 2400485 wikitext text/x-wiki b3tq5y47om1hytp7wbmp86iylhvkctn 2400509 2400485 2026-04-10T10:22:02Z Stashbot 7414 hashar: Updated apitesting job to Quibble 1.17.1 # T422843 T418743 2400509 wikitext text/x-wiki to1prervc2vuv56wjv8kw4s5oxct69r 2400510 2400509 2026-04-10T10:22:24Z Stashbot 7414 hashar: Updated PostgreSQL jobs to Quibble 1.17.1 # T422110 2400510 wikitext text/x-wiki dktp8qpn2vl24zymfosbvkz65zi5wr5 2400511 2400510 2026-04-10T10:24:58Z Stashbot 7414 hashar: Updating all Quibble jobs to 1.17.1 2400511 wikitext text/x-wiki l8mae882v49doy7e1q8kpx73m4oh9pe 0 21478 2400500 2394583 2026-04-10T10:12:11Z Taavi-WMF 41365 fix heading levels 2400500 wikitext text/x-wiki le37rqwbkhse9kwqtemxh9ttf8150u0 0 22503 2400368 2367904 2026-04-09T21:08:33Z DErenrich-WMF 37492 /* Currently reserved values */ add dis1 for discord 2400368 wikitext text/x-wiki in0cg0h4cbccphhklh7icu1cfi3seqb 12 440902 2400370 2391188 2026-04-09T21:12:00Z Nintendofan885 17301 invert poster on dark mode (red is off but looks better) 2400370 wikitext text/x-wiki kvgsiyrjw9jo8sr88nlukh01yyo7isn 498 443628 2400286 2396580 2026-04-09T18:17:08Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24206093216 (https://github.com/cluebotng/component-configs/commits/a97bfe791582e24f1c696f1bd89b965ea233c253) 2400286 wikitext text/x-wiki 4g53ryw46uwii456ttti5bgduf47rdx 498 443947 2400083 2398916 2026-04-09T12:30:55Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-15: deployed 141811e50d (l10n updates: ru) 2400083 wikitext text/x-wiki odze1xp5wb5g2l7hjviy2o00xc6kj7a 0 446037 2400477 2377434 2026-04-10T09:48:01Z MLechvien-WMF 49197 Added the usage of cortobot and more instructions on Follow-up Action Items 2400477 wikitext text/x-wiki 2zwcyisue8cdh6s5k07dioyeita3xv1 0 446977 2400267 2391456 2026-04-09T17:32:26Z APaskulin (WMF) 14880 /* API key management and OAuth integration */ Update scopes 2400267 wikitext text/x-wiki re3a7345a2jjts0over8djo7a5vndv0 0 449160 2400394 2399983 2026-04-10T00:00:17Z Dexbot 30554 Bot: Updating the report 2400394 wikitext text/x-wiki mfq5a2hl93ualy3da02wucl3x09qqrl 2400459 2400394 2026-04-10T09:19:24Z Dexbot 30554 Bot: Updating the report 2400459 wikitext text/x-wiki taa7kkfl4s5hednb3fmumydllf9xn6n 12 451664 2400490 2399945 2026-04-10T10:01:09Z Taavi-WMF 41365 /* Testing locally (optional) */ update for new buildpack stack 2400490 wikitext text/x-wiki eyow4q6ttz2bozky08jdcfhodxlx8fc 828 452251 2400482 2332140 2026-04-10T09:49:45Z Taavi-WMF 41365 add buildpack support 2400482 Scribunto text/plain meu7fpjtbdzyojqfj2i2mm8175608rj 2400488 2400482 2026-04-10T09:57:59Z Taavi 13997 sigh 2400488 Scribunto text/plain 85o5migja6l9od1mt962rtlh9onp5y9 498 452890 2400186 2399537 2026-04-09T14:20:42Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24195131449 (https://github.com/cluebotng/component-configs/commits/6b512f6db7cc4e49078b135e437185906821ae81) 2400186 wikitext text/x-wiki 3rw3kxg11ke37jmjuvanwxmra7vlpn3 2400293 2400186 2026-04-09T18:19:41Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24206093220 (https://github.com/cluebotng/component-configs/commits/a97bfe791582e24f1c696f1bd89b965ea233c253) 2400293 wikitext text/x-wiki pzzv0xnj7vc36jvbejwzz8flap24pm1 2400297 2400293 2026-04-09T18:32:44Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24206712296 (https://github.com/cluebotng/component-configs/commits/e7d5ec988541b9d441a5c565f624b7e88e11204f) 2400297 wikitext text/x-wiki jti3dtwrdl991hggp3soes1lgsb0foe 2400324 2400297 2026-04-09T19:13:33Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24208407251 (https://github.com/cluebotng/component-configs/commits/6cd680dd209bf7fbb01cf24cb6cca82f0fab716d) 2400324 wikitext text/x-wiki ft1zexsaeft3shajv62n347mbsdfbox 2400359 2400324 2026-04-09T20:44:16Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24212235180 (https://github.com/cluebotng/component-configs/commits/1cd21afab7312bd0122c0e735f8f4dca03019011) 2400359 wikitext text/x-wiki 1dosm5rayu06a5z6mejco4gd6faxzpa 2400417 2400359 2026-04-10T01:21:47Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24221486227 (https://github.com/cluebotng/component-configs/commits/d96804861818d7786153d18d47be075a4dbbb6f2) 2400417 wikitext text/x-wiki 528m1njga8w6npp4vf2is9kxw0jf2x3 2400422 2400417 2026-04-10T04:34:34Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24226442413 (https://github.com/cluebotng/component-configs/commits/4f895f83dae3f356cae2a1bbcfea51dd9d18bd15) 2400422 wikitext text/x-wiki t8k7eyo33el0k1stdnf91ks7x4cbwg1 0 455357 2400381 2301051 2026-04-09T22:36:16Z Reedy 9 /* Daemonset */ fix bolding 2400381 wikitext text/x-wiki joyiuthpvy32enib0sou208t2p6pvyu 0 455791 2400336 2384866 2026-04-09T19:59:54Z MPopov (WMF) 5891 /* Data modeling guidelines and recommendations */ 2400336 wikitext text/x-wiki 21bo1j27mmfgzdzyclj2w7tp9yznkua 498 458787 2400285 2396584 2026-04-09T18:16:21Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24206093212 (https://github.com/cluebotng/component-configs/commits/a97bfe791582e24f1c696f1bd89b965ea233c253) 2400285 wikitext text/x-wiki e9dpf8dottguo7h2y18eiuzmzx1y965 0 458823 2400247 2367833 2026-04-09T16:53:14Z Krinkle 44 /* See also */ add [[Sitemap access]] 2400247 wikitext text/x-wiki 53vrchhcrdj34shw2d8m4mst6cnn6hy 498 459243 2400290 2396579 2026-04-09T18:17:55Z Stashbot 7414 wm-bot2: Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/24206093215 (https://github.com/cluebotng/component-configs/commits/a97bfe791582e24f1c696f1bd89b965ea233c253) 2400290 wikitext text/x-wiki 8p1fukoyctxo3xck6w2ox72dbgzbbxe 2400298 2400290 2026-04-09T18:36:09Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24206712245 (https://github.com/cluebotng/component-configs/commits/e7d5ec988541b9d441a5c565f624b7e88e11204f) 2400298 wikitext text/x-wiki g8mg4wb1vukptlo9mofk77vzq8ad4xm 498 459244 2400289 2396581 2026-04-09T18:17:30Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24206093268 (https://github.com/cluebotng/component-configs/commits/a97bfe791582e24f1c696f1bd89b965ea233c253) 2400289 wikitext text/x-wiki n55w9mkj8kyjrscgjwqg9f81hluinmf 498 459255 2400291 2396572 2026-04-09T18:18:35Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24206093239 (https://github.com/cluebotng/component-configs/commits/a97bfe791582e24f1c696f1bd89b965ea233c253) 2400291 wikitext text/x-wiki blegbulegpkv13h5hebc72k69hmc4g6 498 459362 2400292 2396571 2026-04-09T18:18:49Z Stashbot 7414 wm-bot2: Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/24206093210 (https://github.com/cluebotng/component-configs/commits/a97bfe791582e24f1c696f1bd89b965ea233c253) 2400292 wikitext text/x-wiki tcg4dc5ul8keogt4sp639ajek85utrf 0 459699 2400135 2400055 2026-04-09T13:08:58Z Stashbot 7414 jgleeson: civicrm upgraded from d8d3871c to 3d3c0a62 2400135 wikitext text/x-wiki oa5e8bqyqz300cgq2mz5vpv0m09x0av 2400315 2400135 2026-04-09T19:00:50Z Stashbot 7414 larssandergreen: tools upgraded from 986f7f83 to 9bff5f07 2400315 wikitext text/x-wiki 9vmlfl6ueeqde1vyl0lg7kiv3sskvds 2400379 2400315 2026-04-09T21:49:56Z Stashbot 7414 ejegg: fundraising civicrm upgraded from 3d3c0a62 to eb188fa2 2400379 wikitext text/x-wiki npjxzi420ufylalyas8a8ul58bz01ae 2400390 2400379 2026-04-09T23:31:35Z Stashbot 7414 wfan: payments-wiki upgraded from 064a770e to c017d7e7 2400390 wikitext text/x-wiki 2dp0suvnbkm0u3f1gfp3nkfl5eybii6 0 459705 2400251 2377219 2026-04-09T16:59:34Z Krinkle 44 2400251 wikitext text/x-wiki mhdv3zj28y6wcylrvdmh8roa6u0aywn 0 459743 2400335 2398664 2026-04-09T19:59:34Z GModena (WMF) 20194 2400335 wikitext text/x-wiki i9hl8hvxlig6zbv4mxcdemrxw5q676f 0 460018 2400339 2397950 2026-04-09T20:03:27Z GModena (WMF) 20194 2400339 wikitext text/x-wiki tm92sw0merwfl05i8zfdbi5dxme5e6y 828 460036 2400479 2026-04-10T09:48:10Z Taavi-WMF 41365 init 2400479 json application/json mbqvfsr1gawa9eid5wppvmaes442iwf 2400480 2400479 2026-04-10T09:48:19Z Toolforge Image Bot 35765 Bot: Updating available container images 2400480 json application/json k3t1g6it9oz0mvzxt8y50z0i0vbdef1 10 460037 2400484 2026-04-10T09:50:48Z Taavi-WMF 41365 new template 2400484 wikitext text/x-wiki poo1x6epobfscg142a2f7y1djbhq7zv 10 460038 2400486 2026-04-10T09:53:43Z Taavi-WMF 41365 Created page with "{{Documentation subpage}} <!-- Add categories where indicated at the bottom of this page --> == Usage == {{Lua|Module:Toolforge images|Module:Toolforge images/data.json}} <templatedata> { "description": "Display the Toolforge build service container image address", "format": "inline", "params": { "1": { "label": "Image", "type": "string", "suggestedvalues": [ "builder", "runner" ], "default": "builder" }, "2": { "label": "Variant", "..." 2400486 wikitext text/x-wiki i5yr1gefsxhaeldbhbsm8psk14t168z 2 460039 2400524 2026-04-10T11:14:21Z Atsuko-WMF 54082 personal notes/onboarding 2400524 wikitext text/x-wiki 6t866k9kuwedged9sd7kblmgvejb9b6