Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.47.0-wmf.15 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2447911 2447611 2026-08-17T08:03:03Z Stashbot 7414 dcaro: truncate haproxy.log to 1G (out of space) on haproxy-8 2447911 wikitext text/x-wiki lyc14dl1xcke2cdsyct7insk7y6xhp4 0 4108 2447800 2447757 2026-08-16T19:54:06Z ScheduleDeploymentBot 37566 Add [[gerrit:1326077]] to Monday, August 17 UTC late backport window 2447800 wikitext text/x-wiki 5nmmp45yx1jfgnj53ggxl9w58fs0lka 2447802 2447800 2026-08-16T20:03:48Z CMing (WMF) 23277 /* {{Deployment_day|date=2026-08-18}} */ 2447802 wikitext text/x-wiki czcvo7vpm77ytb0hy1uk639no3u726g 2447849 2447802 2026-08-17T07:17:32Z ScheduleDeploymentBot 37566 Add [[gerrit:1216721]] to Monday, August 17 UTC afternoon backport window 2447849 wikitext text/x-wiki c3dkcgs8eaflc7h1o7nulmxz0nahk19 2447913 2447849 2026-08-17T08:06:16Z Nvdtn19 47447 Restore revision [[Special:Permalink/2447802|2447802]] by [[Special:Contributions/CMing (WMF)|CMing (WMF)]] ([[User talk:CMing (WMF)|talk]]) 2447913 wikitext text/x-wiki czcvo7vpm77ytb0hy1uk639no3u726g 2447914 2447913 2026-08-17T08:06:59Z Phuedx (WMF) 47850 /* {{Deployment_day|date=2026-08-17}} */ 2447914 wikitext text/x-wiki 8m8uv4igktaljbm44lav2581jffhip5 2448038 2447914 2026-08-17T10:26:18Z ScheduleDeploymentBot 37566 Add [[gerrit:1311141]] to Monday, August 17 UTC afternoon backport window 2448038 wikitext text/x-wiki o9nkmgcrbnxsggcyhiwhypi9op389st 498 6726 2448104 2447109 2026-08-17T11:38:12Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logging 2448104 wikitext text/x-wiki hmtrtpsvc3rn8stsd3db2rofz9450u5 2448105 2448104 2026-08-17T11:38:15Z Stashbot 7414 tlepage@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logging 2448105 wikitext text/x-wiki 1bru8h0dfk8xkssvwy4hmggr67bk1ct 2448107 2448105 2026-08-17T11:39:48Z Stashbot 7414 tlepage@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logging 2448107 wikitext text/x-wiki skggpt79xfb6fk8e61z344z9zcgw8r1 2448127 2448107 2026-08-17T11:52:14Z Stashbot 7414 tlepage@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logging 2448127 wikitext text/x-wiki p7h9qddo632j7txhd7ud8crjk18z9rx 0 7919 2447809 2447773 2026-08-17T02:00:21Z Stashbot 7414 mwpresync@deploy1003: Started scap build-images: Publishing wmf/next image 2447809 wikitext text/x-wiki 3k5ucdx72enotmyj6ik4nj5smhzpa7e 2447810 2447809 2026-08-17T02:07:09Z Stashbot 7414 mwpresync@deploy1003: Finished scap build-images: Publishing wmf/next image (duration: 06m 47s) 2447810 wikitext text/x-wiki 2gu6movtpqqx4bqwbm6hwchxj5qcfwv 2447813 2447810 2026-08-17T06:06:49Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host gerrit2003.wikimedia.org 2447813 wikitext text/x-wiki 5kulj9qi0ohaegrpbw6ec2j7bls6vtb 2447814 2447813 2026-08-17T06:07:07Z Stashbot 7414 arnaudb@cumin1003: START - Cookbook sre.hosts.reboot-single for host vrts1003.eqiad.wmnet 2447814 wikitext text/x-wiki t9137m7f059ff51plmjkfyy546vdhf0 2447815 2447814 2026-08-17T06:13:24Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host vrts1003.eqiad.wmnet 2447815 wikitext text/x-wiki nrih7ll7xlvy8jq90h8aa9b4ckov6ue 2447816 2447815 2026-08-17T06:13:39Z Stashbot 7414 arnaudb@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host gerrit2003.wikimedia.org 2447816 wikitext text/x-wiki poafffbf04h9xo0pzkps6opun759j1w 2447818 2447816 2026-08-17T06:41:13Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.decommission for hosts cloudvirt1049.eqiad.wmnet 2447818 wikitext text/x-wiki gcm6ibtmnq5hgatr4q3j3ovbu453kkr 2447819 2447818 2026-08-17T06:50:00Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.dns.netbox 2447819 wikitext text/x-wiki 1ruaukvao602ho1fqjgkezlj2kynfm2 2447820 2447819 2026-08-17T06:52:26Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1043.eqiad.wmnet 2447820 wikitext text/x-wiki 2dlchpl3q1j2socv2un345gatvka6ev 2447821 2447820 2026-08-17T06:53:54Z Stashbot 7414 moritzm: installing apr-util security updates 2447821 wikitext text/x-wiki k8nrzgpcsy3k2l4fb0awg5801zoqvi2 2447822 2447821 2026-08-17T06:54:01Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudcumin1001.eqiad.wmnet 2447822 wikitext text/x-wiki a1io5qvrfk1dihxsdm1bv72gwzqechh 2447823 2447822 2026-08-17T06:54:21Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1049.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447823 wikitext text/x-wiki gcq1a5r4hs6cg381iw746augvz54f0v 2447824 2447823 2026-08-17T06:54:46Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1049.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447824 wikitext text/x-wiki muel2yut05nb4jsrxgd8kua0z4cbmw5 2447825 2447824 2026-08-17T06:54:48Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2447825 wikitext text/x-wiki riodln7pohmkq89y9obzfjdndxl96tl 2447826 2447825 2026-08-17T06:54:50Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1049.eqiad.wmnet 2447826 wikitext text/x-wiki b18ohm94g1ta9514i2w2mrigfxb3xj6 2447827 2447826 2026-08-17T06:55:01Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.decommission for hosts cloudvirt1051.eqiad.wmnet 2447827 wikitext text/x-wiki pxcvmsxs4p0k7sl5bf69s14td291uye 2447828 2447827 2026-08-17T06:55:49Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.reboot-single for host cloudcumin2001.codfw.wmnet 2447828 wikitext text/x-wiki l2rpppnw86gi1975lhxwgtvg5z3lw1r 2447829 2447828 2026-08-17T06:59:37Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcumin2001.codfw.wmnet 2447829 wikitext text/x-wiki a202gdw03is6dpsu57kn8ed6j57s5x1 2447830 2447829 2026-08-17T06:59:40Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1043.eqiad.wmnet 2447830 wikitext text/x-wiki 7fgs52lhhc7u2ya59ymm5dthtzqswef 2447831 2447830 2026-08-17T06:59:48Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcumin1001.eqiad.wmnet 2447831 wikitext text/x-wiki joioaioaxljwjdjjuq24em8rcbad5du 2447832 2447831 2026-08-17T06:59:51Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.dns.netbox 2447832 wikitext text/x-wiki qxlmu2my6l2eu6tzfmig5o8btlz3uh8 2447833 2447832 2026-08-17T07:04:06Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1051.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447833 wikitext text/x-wiki flzf4wyk636km1bkqufhq08h88u5oxp 2447834 2447833 2026-08-17T07:05:01Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1043.eqiad.wmnet 2447834 wikitext text/x-wiki 4q765zwdn8lvmtw35lc33490dwpm5sr 2447835 2447834 2026-08-17T07:05:03Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1051.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447835 wikitext text/x-wiki mt77749jni7gimrsingm9jbywrvyho2 2447836 2447835 2026-08-17T07:05:05Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2447836 wikitext text/x-wiki m4xl1ju9r9odrsm6yfsmdxu5sd2qu8n 2447837 2447836 2026-08-17T07:05:08Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1051.eqiad.wmnet 2447837 wikitext text/x-wiki l8zssywlioxnl5g7c54noqczvucdhk4 2447838 2447837 2026-08-17T07:05:20Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.decommission for hosts cloudvirt1054.eqiad.wmnet 2447838 wikitext text/x-wiki gh09ae97xtn14ogy13zfh6goql0lodr 2447839 2447838 2026-08-17T07:05:33Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1043.eqiad.wmnet 2447839 wikitext text/x-wiki fy2wafinprw3i8wxoenxu1q7ji5cugx 2447840 2447839 2026-08-17T07:10:31Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.dns.netbox 2447840 wikitext text/x-wiki fkcm8jspzwgqbuyel2qzrhmx5mq0qgd 2447841 2447840 2026-08-17T07:12:29Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1275 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96098 and previous config saved to /var/cache/conftool/dbconfig/20260817-071225-marostegui.json 2447841 wikitext text/x-wiki 7krmpatpgapk5iilz70fz24wid7rskn 2447842 2447841 2026-08-17T07:13:56Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1275: Pool back 2447842 wikitext text/x-wiki lrq8jhaer4pdj1xaniiextygth71heh 2447843 2447842 2026-08-17T07:15:01Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1054.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447843 wikitext text/x-wiki kgecxbc5a5g39rgi3qa9t9i8payixyr 2447844 2447843 2026-08-17T07:15:30Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1044.eqiad.wmnet 2447844 wikitext text/x-wiki fc723skkqlky4mozgngfk3fhsp9wmj2 2447845 2447844 2026-08-17T07:15:36Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1054.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447845 wikitext text/x-wiki aea926grn9f2rtspdc6137tdbx7ud4z 2447846 2447845 2026-08-17T07:15:39Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2447846 wikitext text/x-wiki dmv32dzi618t87y3m01cchlulhno7ej 2447847 2447846 2026-08-17T07:15:43Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1054.eqiad.wmnet 2447847 wikitext text/x-wiki 4rin8hp4jgwq6tdns1yqdeovat4su6u 2447848 2447847 2026-08-17T07:16:04Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.decommission for hosts cloudvirt1055.eqiad.wmnet 2447848 wikitext text/x-wiki 8mcy95nr1mot5fmzshwqmtnqfig9e5y 2447850 2447848 2026-08-17T07:20:45Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1044.eqiad.wmnet 2447850 wikitext text/x-wiki f9wzswqem7d10yauuhvna4vqcqxt923 2447851 2447850 2026-08-17T07:22:51Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.dns.netbox 2447851 wikitext text/x-wiki gnfibi6qdx9stfarq6uik3nv5d2h4sa 2447853 2447851 2026-08-17T07:24:30Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.clone of db1175.eqiad.wmnet onto db1289.eqiad.wmnet 2447853 wikitext text/x-wiki madj2cinit5q76nb9sjwwccm3ie39i3 2447854 2447853 2026-08-17T07:24:34Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1175: Depool db1175.eqiad.wmnet to then clone it to db1289.eqiad.wmnet - marostegui@cumin1003 2447854 wikitext text/x-wiki rvrta23v8vo9ehpmgi3gz2qw9e6dby3 2447855 2447854 2026-08-17T07:25:00Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1175: Depool db1175.eqiad.wmnet to then clone it to db1289.eqiad.wmnet - marostegui@cumin1003 2447855 wikitext text/x-wiki n03unph1pq4fclhhfkvflq5g98yy8p3 2447857 2447855 2026-08-17T07:25:31Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reboot-single for host idm1001.wikimedia.org 2447857 wikitext text/x-wiki lzf2nxzwqmt1qvrk38cht64p9u6tx2c 2447858 2447857 2026-08-17T07:26:07Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1044.eqiad.wmnet 2447858 wikitext text/x-wiki 50p2km4lw4fq0urgq8234t215r9pyjf 2447859 2447858 2026-08-17T07:26:12Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1044.eqiad.wmnet 2447859 wikitext text/x-wiki 6k9welav5rdoi9qm3bt3arod8hnfxux 2447860 2447859 2026-08-17T07:27:10Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1055.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447860 wikitext text/x-wiki 0gl4jgufytggla8krbdx0xiulxg0dz5 2447861 2447860 2026-08-17T07:28:11Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1055.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447861 wikitext text/x-wiki 2udw0pdxighzvm2pu1ehzwsb7k7zbb4 2447862 2447861 2026-08-17T07:28:14Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2447862 wikitext text/x-wiki e2n74b6cjczjiyo4nbtc5irnte3faah 2447863 2447862 2026-08-17T07:28:16Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1055.eqiad.wmnet 2447863 wikitext text/x-wiki 6sx7hf2itr2ujytbhoya4b2gbiwq8v5 2447864 2447863 2026-08-17T07:28:20Z Stashbot 7414 moritzm: extend the disk of ldap-rw1001 by 80G T331699 2447864 wikitext text/x-wiki 7lh6toeb6t1ngzu6e127c9d4flgkrda 2447865 2447864 2026-08-17T07:28:53Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.decommission for hosts cloudvirt1056.eqiad.wmnet 2447865 wikitext text/x-wiki rr1tmqe9py9hq8dcaphk318t1mk8dp2 2447866 2447865 2026-08-17T07:29:28Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idm1001.wikimedia.org 2447866 wikitext text/x-wiki gw36tn69gshmnmq09v8jjrc7kn8qvai 2447867 2447866 2026-08-17T07:29:44Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reboot-single for host idm-test1001.wikimedia.org 2447867 wikitext text/x-wiki 2i95e7j9ezcr5eki4w8zpamrodwx9z8 2447868 2447867 2026-08-17T07:30:42Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1279 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96102 and previous config saved to /var/cache/conftool/dbconfig/20260817-073038-marostegui.json 2447868 wikitext text/x-wiki a133zp42hf1rrrey95n9eklq5e36ld4 2447869 2447868 2026-08-17T07:31:04Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1279: Pool back 2447869 wikitext text/x-wiki hyuqj7vkjbl2wzzfshnrx8dmxfregtc 2447870 2447869 2026-08-17T07:31:12Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1029.eqiad.wmnet 2447870 wikitext text/x-wiki jqc8oz1dsdlhjg6e8rucyz1edsevrve 2447871 2447870 2026-08-17T07:32:06Z Stashbot 7414 slyngshede@dns1004: START - running authdns-update 2447871 wikitext text/x-wiki onjm708j30cx2wm03zkzhu7mmzc5uhj 2447873 2447871 2026-08-17T07:33:40Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idm-test1001.wikimedia.org 2447873 wikitext text/x-wiki d3zbysowf6esqczy6188j8mqr62pa6o 2447874 2447873 2026-08-17T07:33:56Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1029.eqiad.wmnet 2447874 wikitext text/x-wiki a418af9zrrzwshqg4u9w1tmwg82jc00 2447875 2447874 2026-08-17T07:34:08Z Stashbot 7414 slyngshede@dns1004: END - running authdns-update 2447875 wikitext text/x-wiki j0pue21yaevwpl78uupvnlynxltgl0h 2447876 2447875 2026-08-17T07:34:37Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.dns.netbox 2447876 wikitext text/x-wiki nj8j3hvmy1os3ik0tsvrce8x2vlf04x 2447877 2447876 2026-08-17T07:35:46Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1279 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96104 and previous config saved to /var/cache/conftool/dbconfig/20260817-073542-marostegui.json 2447877 wikitext text/x-wiki klq4jfv9yrhrrs4n68yht3q4f8v8tqg 2447878 2447877 2026-08-17T07:36:13Z Stashbot 7414 slyngshede@cumin1003: START - Cookbook sre.hosts.reboot-single for host idm2001.wikimedia.org 2447878 wikitext text/x-wiki pnsun7muoqo7hiix1127k9gobk8d9ai 2447879 2447878 2026-08-17T07:36:17Z Stashbot 7414 phuedx@deploy1003: Started scap sync-world: Backport for [[gerrit:1216721|viwikivoyage: enable relatedarticle and pop-up (T405724)]] 2447879 wikitext text/x-wiki rdui1chdaktzfa53yo1y1h6kp9z0qf3 2447880 2447879 2026-08-17T07:38:36Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1056.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447880 wikitext text/x-wiki a4i87asi6f6a49brbju5waknkc3oed6 2447881 2447880 2026-08-17T07:39:01Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1056.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447881 wikitext text/x-wiki 6h8ma10rnx37525tq12kfu0zwdmkhpx 2447882 2447881 2026-08-17T07:39:03Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2447882 wikitext text/x-wiki 63hxzokox64pocbhijchlkkrcokg5ip 2447883 2447882 2026-08-17T07:39:06Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1056.eqiad.wmnet 2447883 wikitext text/x-wiki rlcnmmmik1dxsi3i4haibgdg3o379hj 2447884 2447883 2026-08-17T07:40:05Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1029.eqiad.wmnet 2447884 wikitext text/x-wiki sghi64ny9friewbhycea55w17wr958g 2447885 2447884 2026-08-17T07:40:13Z Stashbot 7414 slyngshede@cumin1003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idm2001.wikimedia.org 2447885 wikitext text/x-wiki lme5xqlwebyoa7rb9yrfxqx272m2pd7 2447886 2447885 2026-08-17T07:40:21Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1029.eqiad.wmnet 2447886 wikitext text/x-wiki sxoa4b4vbhugtq09ng9c01zi4p369o8 2447887 2447886 2026-08-17T07:41:31Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.clone of db1174.eqiad.wmnet onto db1288.eqiad.wmnet 2447887 wikitext text/x-wiki l4rqrqb0u5yrlh57b9eooudjgh621y2 2447888 2447887 2026-08-17T07:41:36Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1174: Depool db1174.eqiad.wmnet to then clone it to db1288.eqiad.wmnet - marostegui@cumin1003 2447888 wikitext text/x-wiki omnq5xbggifjgbg276dumh6ttrj8ess 2447889 2447888 2026-08-17T07:41:50Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1174: Depool db1174.eqiad.wmnet to then clone it to db1288.eqiad.wmnet - marostegui@cumin1003 2447889 wikitext text/x-wiki rs2bpugecyuy3eqmfzevoof2byi2piw 2447890 2447889 2026-08-17T07:42:33Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1030.eqiad.wmnet 2447890 wikitext text/x-wiki ekh76u91vy7r66hcdu1sbj15gsy0baj 2447891 2447890 2026-08-17T07:46:36Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1030.eqiad.wmnet 2447891 wikitext text/x-wiki akpde2o6s1ivyt6koab4tc4mn1c6cac 2447892 2447891 2026-08-17T07:47:53Z Stashbot 7414 marostegui@cumin1003: dbctl commit (dc=all): 'Add db1281 to dbctl T407942', diff saved to https://phabricator.wikimedia.org/P96108 and previous config saved to /var/cache/conftool/dbconfig/20260817-074749-marostegui.json 2447892 wikitext text/x-wiki 7k9xygnrglj6shaptq84wiy0mc939ds 2447893 2447892 2026-08-17T07:48:22Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1281: Pool back 2447893 wikitext text/x-wiki hquc36z48a7l6g4b94ldvqts7fyv77s 2447894 2447893 2026-08-17T07:50:51Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.hosts.decommission for hosts cloudvirt1057.eqiad.wmnet 2447894 wikitext text/x-wiki hws6ysu5psghpogwaeppjy4hki6qanu 2447895 2447894 2026-08-17T07:51:44Z Stashbot 7414 tappof: bump space for prometheus k8s-aux in eqiad 2447895 wikitext text/x-wiki pxitjdcb0fa5s7t51oegodun8qapp66 2447896 2447895 2026-08-17T07:52:42Z Stashbot 7414 phuedx@deploy1003: nvdtn19, phuedx: Backport for [[gerrit:1216721|viwikivoyage: enable relatedarticle and pop-up (T405724)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2447896 wikitext text/x-wiki emjizgsc9zfbpnz3vkpeb35gt06lkgq 2447897 2447896 2026-08-17T07:52:45Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1030.eqiad.wmnet 2447897 wikitext text/x-wiki gmawtx1y6kn31hljs4ui9ycvd9lsyow 2447898 2447897 2026-08-17T07:52:50Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1030.eqiad.wmnet 2447898 wikitext text/x-wiki 6dm4ydeyr9725karo53y1nwillwey3p 2447899 2447898 2026-08-17T07:55:17Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1031.eqiad.wmnet 2447899 wikitext text/x-wiki jjxds715yuqtxw2f8yg8jyp8cf9v1pa 2447900 2447899 2026-08-17T07:55:26Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.dns.netbox 2447900 wikitext text/x-wiki 117b2c19k89mi1o3yutl1l5ulvz8wov 2447901 2447900 2026-08-17T07:59:05Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1275: Pool back 2447901 wikitext text/x-wiki r7mbnwcati5t59vk2kqnts9kci5ys7l 2447902 2447901 2026-08-17T07:59:37Z Stashbot 7414 filippo@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1057.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447902 wikitext text/x-wiki a5tok26fv3x684a5w3ad6zv9jihdd0q 2447903 2447902 2026-08-17T08:00:24Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1057.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - filippo@cumin1003" 2447903 wikitext text/x-wiki kzl1upvuo965jvlo4fv9wyvuggm6s5i 2447904 2447903 2026-08-17T08:00:26Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2447904 wikitext text/x-wiki 9tfzocetabjlaispioo1gns85nch2cw 2447905 2447904 2026-08-17T08:00:29Z Stashbot 7414 filippo@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1057.eqiad.wmnet 2447905 wikitext text/x-wiki 9194gf92shodxrzlu4q56ovqe07e8fc 2447906 2447905 2026-08-17T08:00:31Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.clone of db1201.eqiad.wmnet onto db1287.eqiad.wmnet 2447906 wikitext text/x-wiki kvuu1kd29cvv1kaizeq0rj9tooyc214 2447907 2447906 2026-08-17T08:00:34Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1201: Depool db1201.eqiad.wmnet to then clone it to db1287.eqiad.wmnet - marostegui@cumin1003 2447907 wikitext text/x-wiki gjg7hhit3d9xwjd50xuwgk6z456k7os 2447908 2447907 2026-08-17T08:00:58Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1201: Depool db1201.eqiad.wmnet to then clone it to db1287.eqiad.wmnet - marostegui@cumin1003 2447908 wikitext text/x-wiki keen1pop6kiii1canuskq0x329pr8kt 2447909 2447908 2026-08-17T08:02:08Z Stashbot 7414 phuedx@deploy1003: nvdtn19, phuedx: Continuing with deployment 2447909 wikitext text/x-wiki jq9ckfokvpdw0jci2uqxp93lrcbbhzc 2447910 2447909 2026-08-17T08:02:13Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1031.eqiad.wmnet 2447910 wikitext text/x-wiki q0ilczjwu8bwhir3x661o03k71bmncg 2447912 2447910 2026-08-17T08:03:28Z Stashbot 7414 godog: switch production to use dumps-nfs.w.o - T432212 2447912 wikitext text/x-wiki 1wx2m9zqr5wmi82c2dlz7wd4bxv5llj 2447915 2447912 2026-08-17T08:08:37Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1031.eqiad.wmnet 2447915 wikitext text/x-wiki b82zxkpbuboaoaxgwz2mb6155cq03ze 2447916 2447915 2026-08-17T08:08:57Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1031.eqiad.wmnet 2447916 wikitext text/x-wiki dtz4snioh7l7qgrrqo6q9r12wdfu503 2447917 2447916 2026-08-17T08:09:05Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1032.eqiad.wmnet 2447917 wikitext text/x-wiki bi00eyvsdlbqelz3g9m5opp4dvn0j56 2447918 2447917 2026-08-17T08:11:01Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1032.eqiad.wmnet 2447918 wikitext text/x-wiki 8v4vj5mhetan9drgcrwofej7hsp33j0 2447919 2447918 2026-08-17T08:15:28Z Stashbot 7414 phuedx@deploy1003: Finished scap sync-world: Backport for [[gerrit:1216721|viwikivoyage: enable relatedarticle and pop-up (T405724)]] (duration: 39m 12s) 2447919 wikitext text/x-wiki 4ug43z9gv7u51idrsisqmwnx0n10dyu 2447920 2447919 2026-08-17T08:16:14Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1279: Pool back 2447920 wikitext text/x-wiki sko9j8bss1qj011vmy8lr5vfglfqaik 2447921 2447920 2026-08-17T08:17:03Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1032.eqiad.wmnet 2447921 wikitext text/x-wiki cc2aybz0oc2btdrfzya76l7ot27dlj1 2447922 2447921 2026-08-17T08:17:08Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1032.eqiad.wmnet 2447922 wikitext text/x-wiki 4jy9ecj77n5sgngfjlatzrjdo7h5hqy 2447923 2447922 2026-08-17T08:18:43Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1035.eqiad.wmnet 2447923 wikitext text/x-wiki nl62m8qbejtmtpo8o179qaq5jhi69rh 2447924 2447923 2026-08-17T08:19:26Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reimage for host ldap-rw1001.wikimedia.org with OS trixie 2447924 wikitext text/x-wiki qkyj15nznv0dyxzys6ejhpotrcz1ntc 2447925 2447924 2026-08-17T08:21:25Z Stashbot 7414 phuedx@deploy1003: Started scap sync-world: Backport for [[gerrit:1324725|EventStreamConfig: Mark product_metrics.web_base and .web_base_with_ip as Test Kitchen streams (T429898 T430322)]], [[gerrit:1313923|EventStreamConfig: Remove unused web_ui_scroll* streams (T415370)]], [[gerrit:1325546|EventStreamConfig: Remove Watchlist click stream (T434790)]] 2447925 wikitext text/x-wiki n8nhf71v9brw18egzoqofqixj2ormce 2447926 2447925 2026-08-17T08:23:17Z Stashbot 7414 phuedx@deploy1003: bearloga, phuedx: Backport for [[gerrit:1324725|EventStreamConfig: Mark product_metrics.web_base and .web_base_with_ip as Test Kitchen streams (T429898 T430322)]], [[gerrit:1313923|EventStreamConfig: Remove unused web_ui_scroll* streams (T415370)]], [[gerrit:1325546|EventStreamConfig: Remove Watchlist click stream (T434790)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug 2447926 wikitext text/x-wiki f7i8vmvygq0lu15u4kabxjfl3ujz5qv 2447927 2447926 2026-08-17T08:26:28Z Stashbot 7414 phuedx@deploy1003: bearloga, phuedx: Continuing with deployment 2447927 wikitext text/x-wiki 3ph5x3byseksdrxt556yrx7qbutaeci 2447928 2447927 2026-08-17T08:26:52Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1035.eqiad.wmnet 2447928 wikitext text/x-wiki 81wtbdxhvgqt8vzqjf01qr82tdl87bv 2447929 2447928 2026-08-17T08:30:53Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db2209: Maintenance needed 2447929 wikitext text/x-wiki m92ffqu13d1pujjiestjruu2fhkjv0i 2447930 2447929 2026-08-17T08:31:09Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2209: Maintenance needed 2447930 wikitext text/x-wiki 2m9bz8dg6nfoi3d0pibzu2oyfpabxui 2447931 2447930 2026-08-17T08:31:50Z Stashbot 7414 marostegui@cumin1003: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db2209.codfw.wmnet with reason: Maintenance 2447931 wikitext text/x-wiki d7oj066purrix97cirpadif2189w80g 2447932 2447931 2026-08-17T08:32:47Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.downtime for 2:00:00 on ldap-rw1001.wikimedia.org with reason: host reimage 2447932 wikitext text/x-wiki 2v6pppjjnyelwv56yvtmfimlm3bejih 2447933 2447932 2026-08-17T08:33:31Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1281: Pool back 2447933 wikitext text/x-wiki 5482ksm1bx2v4darr73eea9gz0vv83i 2447934 2447933 2026-08-17T08:34:12Z Stashbot 7414 phuedx@deploy1003: Finished scap sync-world: Backport for [[gerrit:1324725|EventStreamConfig: Mark product_metrics.web_base and .web_base_with_ip as Test Kitchen streams (T429898 T430322)]], [[gerrit:1313923|EventStreamConfig: Remove unused web_ui_scroll* streams (T415370)]], [[gerrit:1325546|EventStreamConfig: Remove Watchlist click stream (T434790)]] (duration: 12m 42s) 2447934 wikitext text/x-wiki 82dtabqqs6hkxwwvauvfhce8po8uyl9 2447935 2447934 2026-08-17T08:34:34Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1035.eqiad.wmnet 2447935 wikitext text/x-wiki l4b1i05kp5867b64ow3oxa9wq3fpwh8 2447936 2447935 2026-08-17T08:34:56Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1035.eqiad.wmnet 2447936 wikitext text/x-wiki egcho55hoxjgerlgnd80g2vzhw7taix 2447937 2447936 2026-08-17T08:35:34Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1053.eqiad.wmnet 2447937 wikitext text/x-wiki 1ahd7wqcvrom3c5dtpy4m2v4wln81mv 2447938 2447937 2026-08-17T08:35:44Z Stashbot 7414 phuedx: UTC morning backport window done 2447938 wikitext text/x-wiki nbrww96bmigboyhm6wpmgih6br23pai 2447939 2447938 2026-08-17T08:36:47Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.decommission 2447939 wikitext text/x-wiki 4805vuwu644idcrigfssn5zf76mmors 2447940 2447939 2026-08-17T08:36:57Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db1152.eqiad.wmnet 2447940 wikitext text/x-wiki qke7tkqa0liyrofndbd7hqt5f71a888 2447941 2447940 2026-08-17T08:37:58Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1053.eqiad.wmnet 2447941 wikitext text/x-wiki 7yskz6an2p3vg8k2jqsj7dqpk65aebo 2447942 2447941 2026-08-17T08:38:35Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ldap-rw1001.wikimedia.org with reason: host reimage 2447942 wikitext text/x-wiki 14kyylue09m76q3zdzyuamquzwxec8h 2447943 2447942 2026-08-17T08:42:36Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.dns.netbox 2447943 wikitext text/x-wiki 0n5ro19zep0olby15z3o86mxicp7tap 2447944 2447943 2026-08-17T08:43:08Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1053.eqiad.wmnet 2447944 wikitext text/x-wiki b1wqe9vd6oo3rc45p91jyzybguemzqj 2447945 2447944 2026-08-17T08:43:13Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1053.eqiad.wmnet 2447945 wikitext text/x-wiki a6u5ehs1s8tuw4wrdlmv8x92asf7da5 2447946 2447945 2026-08-17T08:46:04Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1054.eqiad.wmnet 2447946 wikitext text/x-wiki mu5rac4antk5q7dot8b8hf1siisyk6z 2447947 2447946 2026-08-17T08:46:19Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db1152.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003" 2447947 wikitext text/x-wiki s9fhow2bj2n35eq75io7bs3uu97wpz0 2447948 2447947 2026-08-17T08:46:25Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1201: Pool db1201.eqiad.wmnet in after cloning 2447948 wikitext text/x-wiki qgiskesblkjii4rve0xpj5tjs1ay7zu 2447949 2447948 2026-08-17T08:46:36Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db1152.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003" 2447949 wikitext text/x-wiki ey7z97vedpvv8xy5yde9ej9vfyuevy7 2447950 2447949 2026-08-17T08:46:39Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2447950 wikitext text/x-wiki i9i64nqbxwufwr52nlac9jn8upyxcgo 2447951 2447950 2026-08-17T08:46:42Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db1152.eqiad.wmnet 2447951 wikitext text/x-wiki bqwa8yzykjnus0b7ot9oas0fhq9bt21 2447952 2447951 2026-08-17T08:46:47Z Stashbot 7414 marostegui@cumin1003: Removing db1152 from zarcillo T434480 2447952 wikitext text/x-wiki ahrh1nt9ab2ta43w48ilapz874uyig6 2447953 2447952 2026-08-17T08:46:50Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.decommission (exit_code=0) 2447953 wikitext text/x-wiki pb0s5zv9bgpxywrxy92a922x2g7bxny 2447954 2447953 2026-08-17T08:47:31Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1175: Pool db1175.eqiad.wmnet in after cloning 2447954 wikitext text/x-wiki q0ourvq8v8gisbxl7s4oumxikqrumwr 2447955 2447954 2026-08-17T08:48:44Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1054.eqiad.wmnet 2447955 wikitext text/x-wiki hs2fjg3y39cqvbk9dyjmat5l6jaf0y3 2447956 2447955 2026-08-17T08:54:07Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1054.eqiad.wmnet 2447956 wikitext text/x-wiki js3rmchi023d7xl5z0wlve36e6bswi7 2447957 2447956 2026-08-17T08:54:24Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1054.eqiad.wmnet 2447957 wikitext text/x-wiki sdcc6gg9czghvntnmy6pme7etkaxgpr 2447959 2447957 2026-08-17T08:57:43Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1174: Pool db1174.eqiad.wmnet in after cloning 2447959 wikitext text/x-wiki shkka9e27zd5gq8ormbkfbphox4lm7f 2447960 2447959 2026-08-17T09:01:22Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ldap-rw1001.wikimedia.org with OS trixie 2447960 wikitext text/x-wiki f3oaecktqkcel772l3q2jz3xi2u6yvr 2447961 2447960 2026-08-17T09:15:20Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db1901.eqiad.wmnet 2447961 wikitext text/x-wiki fm0mroyj8eai4dl6w0iz9yk4kzyg25m 2447962 2447961 2026-08-17T09:15:23Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2447962 wikitext text/x-wiki 8zwermmnkurq72itnapfo23bvdapjmz 2447963 2447962 2026-08-17T09:26:52Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db1901.eqiad.wmnet - fceratto@cumin1003" 2447963 wikitext text/x-wiki 1pfzj2lwz6hpyt90w7kwc719wag4uqo 2447964 2447963 2026-08-17T09:26:56Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db1901.eqiad.wmnet - fceratto@cumin1003" 2447964 wikitext text/x-wiki 34g6x0ccd2igint8gatis9q3vzsls9a 2447965 2447964 2026-08-17T09:26:59Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2447965 wikitext text/x-wiki aq0bqg22nhudq7gtufskwi92ekrtzhx 2447966 2447965 2026-08-17T09:27:02Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.wipe-cache db1901.eqiad.wmnet on all recursors 2447966 wikitext text/x-wiki 0ni071wdgb909aewufnyq0yit1rs42h 2447967 2447966 2026-08-17T09:27:04Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) db1901.eqiad.wmnet on all recursors 2447967 wikitext text/x-wiki 2pzz8k2bkg19ic3dlughyfu7baqesbn 2447968 2447967 2026-08-17T09:27:29Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM db1901.eqiad.wmnet - fceratto@cumin1003" 2447968 wikitext text/x-wiki l9iicxci2jsl0tn3wiujx5r8wrxq35l 2447969 2447968 2026-08-17T09:27:33Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM db1901.eqiad.wmnet - fceratto@cumin1003" 2447969 wikitext text/x-wiki 7puo65n43a6h75ffjdh160y078skt5s 2447970 2447969 2026-08-17T09:28:19Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.reimage for host db1901.eqiad.wmnet with OS trixie 2447970 wikitext text/x-wiki dmkamolkg4ydoyajqku1gvql54zi9u1 2447971 2447970 2026-08-17T09:31:39Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1201: Pool db1201.eqiad.wmnet in after cloning 2447971 wikitext text/x-wiki cn2gcyloq83el2wnnwk90n9nz6hz064 2447972 2447971 2026-08-17T09:31:47Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1201.eqiad.wmnet onto db1287.eqiad.wmnet 2447972 wikitext text/x-wiki d0sxo1a4s1ixjntd2rbetvtpp873w5d 2447973 2447972 2026-08-17T09:32:45Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1175: Pool db1175.eqiad.wmnet in after cloning 2447973 wikitext text/x-wiki s2puvxgpaewwvyz6p9zcj9j0f4cr5eb 2447974 2447973 2026-08-17T09:33:47Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1175.eqiad.wmnet onto db1289.eqiad.wmnet 2447974 wikitext text/x-wiki brp7aw5sk665rrkjfgxk7spucgaj2qn 2447975 2447974 2026-08-17T09:39:22Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.clone of db1172.eqiad.wmnet onto db1286.eqiad.wmnet 2447975 wikitext text/x-wiki qr39pfx27v7x368m74dofkrrsdx1518 2447976 2447975 2026-08-17T09:39:26Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.depool depool db1172: Depool db1172.eqiad.wmnet to then clone it to db1286.eqiad.wmnet - marostegui@cumin1003 2447976 wikitext text/x-wiki f41uz8bri8w4k688rbnvwplgd8uom2x 2447977 2447976 2026-08-17T09:40:21Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1172: Depool db1172.eqiad.wmnet to then clone it to db1286.eqiad.wmnet - marostegui@cumin1003 2447977 wikitext text/x-wiki 34jqx2c32w6klsgw30u6nd2t69spqip 2447978 2447977 2026-08-17T09:42:57Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1174: Pool db1174.eqiad.wmnet in after cloning 2447978 wikitext text/x-wiki hftfgkue5oeyer0dr4rs5rro7bnr7zs 2447979 2447978 2026-08-17T09:43:06Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1174.eqiad.wmnet onto db1288.eqiad.wmnet 2447979 wikitext text/x-wiki 8g746i9xav4p53dfaoypgxiwsraqgmx 2447980 2447979 2026-08-17T09:43:44Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.reboot-nodes rolling reboot on A:wikikube-worker-codfw 2447980 wikitext text/x-wiki jd08pg71br3u4frqnpx8q7yrbyhup66 2447981 2447980 2026-08-17T09:44:15Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2002,2005-2006,2011-2015,2017-2018,2033-2037].codfw.wmnet 2447981 wikitext text/x-wiki 1dh7ioi7bj8l8rowe6ynpxa3dq67q9l 2447982 2447981 2026-08-17T09:48:39Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2447982 wikitext text/x-wiki 00s6bvda6v63zttih4x7bbvv3aknfpv 2447983 2447982 2026-08-17T09:48:43Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db1902.eqiad.wmnet 2447983 wikitext text/x-wiki 2gorev5k8gvno9grkbo6a5fe6bcwmb4 2447984 2447983 2026-08-17T09:48:45Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2447984 wikitext text/x-wiki tw3j32wyx5c7dxd52fis3lqvng8w82h 2447985 2447984 2026-08-17T09:48:59Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db1903.eqiad.wmnet 2447985 wikitext text/x-wiki c2pkwn6uw41zds7w65h5bya0o5jnr2k 2447986 2447985 2026-08-17T09:50:06Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2447986 wikitext text/x-wiki ovk55nohzp4y5viokgtopyqikzun4pz 2447987 2447986 2026-08-17T09:53:00Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2002,2005-2006,2011-2015,2017-2018,2033-2037].codfw.wmnet 2447987 wikitext text/x-wiki 0k4i2f1ltbah2adk8o8emc6gil9ecco 2447989 2447987 2026-08-17T09:53:57Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2447989 wikitext text/x-wiki 3er9rptw9u2710sl4s2pclwdkingjcr 2447990 2447989 2026-08-17T09:56:56Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1326227|Enable desktop/native lazy loading everywhere (T148047)]] 2447990 wikitext text/x-wiki m5akqfzp45dcwfp7l4kaj3puwfwx31d 2447991 2447990 2026-08-17T09:58:48Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1326227|Enable desktop/native lazy loading everywhere (T148047)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2447991 wikitext text/x-wiki nhzqvb7n771wr9dty78zmyta380by92 2447992 2447991 2026-08-17T09:59:49Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1055.eqiad.wmnet 2447992 wikitext text/x-wiki jmpcatcu4hmurllnk2rga2kbnlhrlc9 2447993 2447992 2026-08-17T09:59:52Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2447993 wikitext text/x-wiki duok7okjxlii3ga2v8ykk7i4leuyodt 2447994 2447993 2026-08-17T10:00:35Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2447994 wikitext text/x-wiki d2c6ubpikhmuxuenxu5l4shbfso5t40 2447995 2447994 2026-08-17T10:00:54Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2447995 wikitext text/x-wiki fol994yc97tfyp3xuz6byw9g7pbujjo 2447996 2447995 2026-08-17T10:01:01Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2002,2005-2006,2011-2015,2017-2018,2033-2037].codfw.wmnet 2447996 wikitext text/x-wiki nb0f1qfg7pf2b8wcig5gpbiqglf1ftt 2447997 2447996 2026-08-17T10:01:04Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2002,2005-2006,2011-2015,2017-2018,2033-2037].codfw.wmnet 2447997 wikitext text/x-wiki fszxyh3fozou809997fdit3xcmbrziz 2447998 2447997 2026-08-17T10:01:40Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2038-2039,2041-2042,2044,2046,2049-2051,2055-2060].codfw.wmnet 2447998 wikitext text/x-wiki hk03xgbyfkn41718xsao56kyn266tyh 2447999 2447998 2026-08-17T10:01:44Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db1903.eqiad.wmnet - fceratto@cumin1003" 2447999 wikitext text/x-wiki d3u0umwijo6oyumzjmogdnp23j3voxr 2448000 2447999 2026-08-17T10:03:06Z Stashbot 7414 blake@deploy1003: helmfile [codfw] START helmfile.d/services/proton: apply 2448000 wikitext text/x-wiki s1l8nnx8uz5zm1wl8ewzu7gdrn43aq7 2448001 2448000 2026-08-17T10:03:26Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448001 wikitext text/x-wiki n9mdvraoziabq1fcxrcilcj140pcn1k 2448002 2448001 2026-08-17T10:03:28Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.wipe-cache db1902.eqiad.wmnet on all recursors 2448002 wikitext text/x-wiki 3xnw3jenjrf47x7tt3dkga1e6vt8ntc 2448003 2448002 2026-08-17T10:03:31Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) db1902.eqiad.wmnet on all recursors 2448003 wikitext text/x-wiki cs02a1qbosxg7j7khfmee67qp2bjz0j 2448004 2448003 2026-08-17T10:04:10Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326227|Enable desktop/native lazy loading everywhere (T148047)]] (duration: 07m 13s) 2448004 wikitext text/x-wiki 5fbgws9vd2sblfoer9aoj0mfz7iqp6c 2448005 2448004 2026-08-17T10:04:55Z Stashbot 7414 blake@deploy1003: helmfile [codfw] DONE helmfile.d/services/proton: apply 2448005 wikitext text/x-wiki bzq41fsak21f7o1762r6psely7nx3vp 2448006 2448005 2026-08-17T10:07:02Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM db1902.eqiad.wmnet - fceratto@cumin1003" 2448006 wikitext text/x-wiki lf8o1u5n7mxm12h3rfqwd7bzuxp6eeu 2448007 2448006 2026-08-17T10:07:32Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM db1902.eqiad.wmnet - fceratto@cumin1003" 2448007 wikitext text/x-wiki hvsulgc8vhandcdbe1pjslw7mqgdaiy 2448008 2448007 2026-08-17T10:07:44Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.reimage for host db1902.eqiad.wmnet with OS trixie 2448008 wikitext text/x-wiki 7d47r6wdh9aei5f477ussiowdqhfhfv 2448009 2448008 2026-08-17T10:08:22Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db1903.eqiad.wmnet - fceratto@cumin1003" 2448009 wikitext text/x-wiki sjtsye50h2arlns0hbcbdptcgoda5y0 2448010 2448009 2026-08-17T10:08:24Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2448010 wikitext text/x-wiki miz0sajzvpnn36hh60zptljwxn4aajl 2448011 2448010 2026-08-17T10:08:26Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448011 wikitext text/x-wiki pg16dc0uc3mdyhj4lbhctzww1ti7mre 2448012 2448011 2026-08-17T10:08:35Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host db1901.eqiad.wmnet with OS trixie 2448012 wikitext text/x-wiki los5c5l49owzpr51d5wae0qxh0ftsuz 2448013 2448012 2026-08-17T10:08:38Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=97) for new host db1901.eqiad.wmnet 2448013 wikitext text/x-wiki 72nwesceoh88e7pmo6i98by2m10k1r8 2448014 2448013 2026-08-17T10:09:46Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1055.eqiad.wmnet 2448014 wikitext text/x-wiki 3ehhtvrq7cdw5vmlghttm774ozjcn3e 2448015 2448014 2026-08-17T10:10:28Z Stashbot 7414 moritzm: installing unzip security updates 2448015 wikitext text/x-wiki qecshc2qi2i5gb3op1q34wa3v91r55e 2448016 2448015 2026-08-17T10:10:39Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2038-2039,2041-2042,2044,2046,2049-2051,2055-2060].codfw.wmnet 2448016 wikitext text/x-wiki 8c5r5o22nlgmtnzv6kguixtecp4i1sv 2448017 2448016 2026-08-17T10:10:54Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448017 wikitext text/x-wiki 391ndhldhnhbbhf4h1xongueto4pn50 2448018 2448017 2026-08-17T10:10:57Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.wipe-cache db1903.eqiad.wmnet on all recursors 2448018 wikitext text/x-wiki hbsebanjz4003ze7ml86oc82okvr64k 2448019 2448018 2026-08-17T10:11:00Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) db1903.eqiad.wmnet on all recursors 2448019 wikitext text/x-wiki 7aq7hbfyfj35tuh8a7flsh7r6ubgsq1 2448020 2448019 2026-08-17T10:11:14Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448020 wikitext text/x-wiki mh7mxrxu7fw0t49esbsu9511onm2aln 2448021 2448020 2026-08-17T10:13:06Z Stashbot 7414 Amir1: mwscript-k8s --dblist=all -- purgeUserOptions.php --login-age 5 uls-preferences (T406724) 2448021 wikitext text/x-wiki ln3262velwzo9zwjrtrxxndib51zclz 2448022 2448021 2026-08-17T10:15:13Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1055.eqiad.wmnet 2448022 wikitext text/x-wiki ddcet1fx0258anij6v8193judutwxf2 2448023 2448022 2026-08-17T10:15:18Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1055.eqiad.wmnet 2448023 wikitext text/x-wiki jtw3jq7zj88djk7v585n5d9yay2erz4 2448024 2448023 2026-08-17T10:18:50Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2038-2039,2041-2042,2044,2046,2049-2051,2055-2060].codfw.wmnet 2448024 wikitext text/x-wiki qit5liy4v88d5bncnpxudtewldds451 2448025 2448024 2026-08-17T10:18:53Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2038-2039,2041-2042,2044,2046,2049-2051,2055-2060].codfw.wmnet 2448025 wikitext text/x-wiki mjwfephw9ni8t5zd5t6murlc7xagr5d 2448026 2448025 2026-08-17T10:19:26Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2061-2062,2064-2065,2067-2077].codfw.wmnet 2448026 wikitext text/x-wiki ctquxrsob2m1vaqmzkj2x98572a2fky 2448027 2448026 2026-08-17T10:20:02Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1056.eqiad.wmnet 2448027 wikitext text/x-wiki 0nxx72eceoqukwirif8x12ebmtvguhv 2448028 2448027 2026-08-17T10:24:08Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2448028 wikitext text/x-wiki 7mke1hu90j4a9my9yy43lfp3932vzkn 2448029 2448028 2026-08-17T10:24:10Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=93) for new host db1903.eqiad.wmnet 2448029 wikitext text/x-wiki e9q2ncyvne8ryf3wsni6gsifn0f3gqt 2448030 2448029 2026-08-17T10:24:20Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host db1902.eqiad.wmnet with OS trixie 2448030 wikitext text/x-wiki 1fjq1knmeeojtyxbogusu0jglanenrg 2448031 2448030 2026-08-17T10:24:22Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=97) for new host db1902.eqiad.wmnet 2448031 wikitext text/x-wiki irdl4ax5wgcywxv150pc8b01y40x1oy 2448032 2448031 2026-08-17T10:24:29Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db1902.eqiad.wmnet 2448032 wikitext text/x-wiki b2xqerbvvhxn6qpgitfvthh8hsecxmj 2448033 2448032 2026-08-17T10:24:31Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448033 wikitext text/x-wiki gd6rtzjekz6vqkknkh29efgbe6k9kdd 2448034 2448033 2026-08-17T10:25:26Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.dns.netbox (exit_code=97) 2448034 wikitext text/x-wiki qcjqcqgufr7ks54zivm0rvmpfho025c 2448035 2448034 2026-08-17T10:25:30Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host db1902.eqiad.wmnet 2448035 wikitext text/x-wiki f2g17eq0drp0koq70t2xk955izrn8ep 2448036 2448035 2026-08-17T10:25:43Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db2901.codfw.wmnet 2448036 wikitext text/x-wiki 41rstt8bmdlsomzwzbls92h6770racg 2448037 2448036 2026-08-17T10:25:46Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448037 wikitext text/x-wiki dos1qaing2zs757aqp0cil501l1zsz9 2448039 2448037 2026-08-17T10:28:24Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2061-2062,2064-2065,2067-2077].codfw.wmnet 2448039 wikitext text/x-wiki sdink51lajy180bgmwpydhqxlffj2yb 2448040 2448039 2026-08-17T10:32:30Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1056.eqiad.wmnet 2448040 wikitext text/x-wiki exytcg9yni0p6awnzytmepka6vv31yc 2448041 2448040 2026-08-17T10:36:45Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2061-2062,2064-2065,2067-2077].codfw.wmnet 2448041 wikitext text/x-wiki okgwh62fkqx9h7429sonnm1fh7x1oel 2448042 2448041 2026-08-17T10:36:48Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2061-2062,2064-2065,2067-2077].codfw.wmnet 2448042 wikitext text/x-wiki o6390n8cvzz8ys6appk8kn9wuynofze 2448043 2448042 2026-08-17T10:37:19Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2078,2087-2095,2102-2106].codfw.wmnet 2448043 wikitext text/x-wiki qy8q0348dha22gb95qswkm35w0x2hf1 2448044 2448043 2026-08-17T10:38:33Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1056.eqiad.wmnet 2448044 wikitext text/x-wiki b40x08n9agmyn008tpqoc8c9dfj3wyq 2448045 2448044 2026-08-17T10:38:38Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1056.eqiad.wmnet 2448045 wikitext text/x-wiki dbghm5w84ud8c7mxjag9xgk2ygg6eo2 2448046 2448045 2026-08-17T10:38:56Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1057.eqiad.wmnet 2448046 wikitext text/x-wiki h0hnbebfwu31wiu6ka59znln717vkvd 2448047 2448046 2026-08-17T10:45:21Z Stashbot 7414 blake@deploy1003: helmfile [codfw] START helmfile.d/services/proton: apply 2448047 wikitext text/x-wiki p5hue0ti2u164u6i3pjkl3s62ndgzr7 2448048 2448047 2026-08-17T10:45:46Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2078,2087-2095,2102-2106].codfw.wmnet 2448048 wikitext text/x-wiki qczf4cly28pyil7vgk3hallzpbo7omm 2448049 2448048 2026-08-17T10:46:08Z Stashbot 7414 blake@deploy1003: helmfile [codfw] DONE helmfile.d/services/proton: apply 2448049 wikitext text/x-wiki f86kg6dn49g1hhd9zuwfwa9w8v03eir 2448050 2448049 2026-08-17T10:46:23Z Stashbot 7414 jmm@deploy1003: helmfile [staging] START helmfile.d/services/proton: apply 2448050 wikitext text/x-wiki ten3vr6ydcy68cimbm9qf9xjvm7omk3 2448051 2448050 2026-08-17T10:47:13Z Stashbot 7414 jmm@deploy1003: helmfile [staging] DONE helmfile.d/services/proton: apply 2448051 wikitext text/x-wiki pu6atpjyjusjog2xzy3bsu92iuotuuo 2448052 2448051 2026-08-17T10:54:12Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2078,2087-2095,2102-2106].codfw.wmnet 2448052 wikitext text/x-wiki rpu3okod9gl3qqdrwjz4tn45h1i21eh 2448053 2448052 2026-08-17T10:54:16Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2078,2087-2095,2102-2106].codfw.wmnet 2448053 wikitext text/x-wiki dgc7k5lumlgtqhnr9wqbvllvvpma6cb 2448054 2448053 2026-08-17T10:54:54Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2107-2115,2124-2129].codfw.wmnet 2448054 wikitext text/x-wiki k0q17tnf38wvzkleary5eqnlka2yrb8 2448055 2448054 2026-08-17T10:57:21Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.pool pool db1172: Pool db1172.eqiad.wmnet in after cloning 2448055 wikitext text/x-wiki fjsf9gtzrfa3wn465v6g54er1jhbiga 2448056 2448055 2026-08-17T11:03:17Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2107-2115,2124-2129].codfw.wmnet 2448056 wikitext text/x-wiki s3jrdgy210p2gd6ckp91o7mfs7i0nqn 2448057 2448056 2026-08-17T11:06:31Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db2901.codfw.wmnet - fceratto@cumin1003" 2448057 wikitext text/x-wiki 0jqz815tdfc5m4d9us84his9ounr5rf 2448058 2448057 2026-08-17T11:10:56Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2448058 wikitext text/x-wiki ji11iswxk1euuauwpmzrlhv8fmpku0j 2448059 2448058 2026-08-17T11:11:06Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2448059 wikitext text/x-wiki 8ha69z2e8vobebxbrnymuh57rkp6afw 2448060 2448059 2026-08-17T11:11:17Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] START helmfile.d/services/proton: apply 2448060 wikitext text/x-wiki 17mmjo0n4tiqmypc1dsiw4fu56zpild 2448061 2448060 2026-08-17T11:11:29Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2448061 wikitext text/x-wiki tkubfe7gk0zu1ntibiux3zbs6e4pbg1 2448062 2448061 2026-08-17T11:11:35Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2448062 wikitext text/x-wiki ag27wgt8yfjvmpd1pnn5s14uu0d8rio 2448063 2448062 2026-08-17T11:11:38Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2107-2115,2124-2129].codfw.wmnet 2448063 wikitext text/x-wiki ker91czbhvpflnan8x3norsn70x4ck4 2448064 2448063 2026-08-17T11:11:41Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2107-2115,2124-2129].codfw.wmnet 2448064 wikitext text/x-wiki dptx4fnxtl023nt7e75rgr4mma2123d 2448065 2448064 2026-08-17T11:12:06Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2130-2143].codfw.wmnet 2448065 wikitext text/x-wiki og5mgeq66pn3bcuib00mnx0ps0wya57 2448066 2448065 2026-08-17T11:12:13Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM db2901.codfw.wmnet - fceratto@cumin1003" 2448066 wikitext text/x-wiki q1479plxqiwr5w2lbu8woe801vletjl 2448067 2448066 2026-08-17T11:12:15Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448067 wikitext text/x-wiki m4s0xtqmea0amgw3nqkici0ha5po0tk 2448068 2448067 2026-08-17T11:12:17Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.wipe-cache db2901.codfw.wmnet on all recursors 2448068 wikitext text/x-wiki hzesmwdqk3696fo28oppmvi38f05hd8 2448069 2448068 2026-08-17T11:12:19Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) db2901.codfw.wmnet on all recursors 2448069 wikitext text/x-wiki hzbhfvu874w3vy98mi2u3oxkpxea1rn 2448070 2448069 2026-08-17T11:12:49Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM db2901.codfw.wmnet - fceratto@cumin1003" 2448070 wikitext text/x-wiki mm3r9g8zea600hbj5cnn2oz0ikmgbck 2448071 2448070 2026-08-17T11:12:54Z Stashbot 7414 fceratto@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM db2901.codfw.wmnet - fceratto@cumin1003" 2448071 wikitext text/x-wiki a9czyw60ni3rcqjs3m15uo7gluz55gr 2448072 2448071 2026-08-17T11:13:08Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.hosts.reimage for host db2901.codfw.wmnet with OS trixie 2448072 wikitext text/x-wiki 4bgw100j78mnoyvbgwyj60g1sou6oqc 2448073 2448072 2026-08-17T11:13:12Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] START helmfile.d/services/thumbor: apply 2448073 wikitext text/x-wiki semzrpflioqxbjm4228ce1963ucgl3y 2448074 2448073 2026-08-17T11:14:09Z Stashbot 7414 ladsgroup@deploy1003: helmfile [staging] DONE helmfile.d/services/thumbor: apply 2448074 wikitext text/x-wiki n8dwfum1j984j7in9dmp2b8or60katq 2448075 2448074 2026-08-17T11:15:32Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] START helmfile.d/services/thumbor: apply 2448075 wikitext text/x-wiki h948mo90ufgsytzx7r6t83lxz4cxa85 2448076 2448075 2026-08-17T11:15:59Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1057.eqiad.wmnet 2448076 wikitext text/x-wiki irz0fp8fmntdu449zuopmgy4uxpyvm6 2448077 2448076 2026-08-17T11:16:50Z Stashbot 7414 ladsgroup@deploy1003: helmfile [codfw] DONE helmfile.d/services/thumbor: apply 2448077 wikitext text/x-wiki nj7vjykw10t2w8w4nqbb6h2sxlzthqa 2448078 2448077 2026-08-17T11:17:12Z Stashbot 7414 jmm@deploy1003: helmfile [codfw] DONE helmfile.d/services/proton: apply 2448078 wikitext text/x-wiki 125pbsswt22qhes8vcwucu6x57fxg8l 2448079 2448078 2026-08-17T11:17:24Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] START helmfile.d/services/proton: apply 2448079 wikitext text/x-wiki o5cuksv3iavxzbvbwosz7lqqvkgusq9 2448080 2448079 2026-08-17T11:18:42Z Stashbot 7414 jmm@deploy1003: helmfile [eqiad] DONE helmfile.d/services/proton: apply 2448080 wikitext text/x-wiki h50dv6z9dzil29uojob3zeo87whq9v0 2448081 2448080 2026-08-17T11:19:04Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1326249|ImagePage: move TOC element below file link (T332644)]] 2448081 wikitext text/x-wiki h8lt648xooxc321w4eh0rhxsxbo0l9h 2448082 2448081 2026-08-17T11:19:52Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2130-2143].codfw.wmnet 2448082 wikitext text/x-wiki cayc2d8lzhhfyz7zkdchuam0dr84brt 2448083 2448082 2026-08-17T11:20:31Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] START helmfile.d/services/thumbor: apply 2448083 wikitext text/x-wiki 3secf2081841lon8150190a0ayz2vvp 2448084 2448083 2026-08-17T11:21:22Z Stashbot 7414 ladsgroup@deploy1003: helmfile [eqiad] DONE helmfile.d/services/thumbor: apply 2448084 wikitext text/x-wiki lra63zpag5vrmyz4wejoxsmr46u62m6 2448085 2448084 2026-08-17T11:21:41Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1057.eqiad.wmnet 2448085 wikitext text/x-wiki 8xwsecbq1ye8cloy8xu5zsd0t142jho 2448086 2448085 2026-08-17T11:21:47Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1057.eqiad.wmnet 2448086 wikitext text/x-wiki 8m9a492nhg8luhbos93s6lr8jtctole 2448087 2448086 2026-08-17T11:22:07Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1326249|ImagePage: move TOC element below file link (T332644)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2448087 wikitext text/x-wiki 5dadrkgbcnbe7g4ldvttx2q49smg5us 2448088 2448087 2026-08-17T11:22:33Z Stashbot 7414 jayme@deploy1003: helmfile [eqiad] START helmfile.d/admin 'sync'. 2448088 wikitext text/x-wiki j06rxeumx0fvwdl8cue358kb8iy1eqn 2448089 2448088 2026-08-17T11:22:36Z Stashbot 7414 jayme@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'sync'. 2448089 wikitext text/x-wiki rwp6e800u4fw5fouo62k9yw26wh91sm 2448090 2448089 2026-08-17T11:22:48Z Stashbot 7414 jayme@deploy1003: helmfile [eqiad] START helmfile.d/admin 'sync'. 2448090 wikitext text/x-wiki szodpphts9gd5wxfhbqafe72r6wdkrc 2448091 2448090 2026-08-17T11:23:46Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with deployment 2448091 wikitext text/x-wiki bvdjt1kouiyp4awnsz46mfj77fuazv8 2448092 2448091 2026-08-17T11:26:11Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1058.eqiad.wmnet 2448092 wikitext text/x-wiki fn822xz3c8nib29522qeglh8qwkyy34 2448093 2448092 2026-08-17T11:28:13Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2130-2143].codfw.wmnet 2448093 wikitext text/x-wiki cew60c550rk6dutz9njgwo29x5ndgle 2448094 2448093 2026-08-17T11:28:16Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2130-2143].codfw.wmnet 2448094 wikitext text/x-wiki 0cwhkj8ulszthm1nbp2axsxj1ysgpxt 2448095 2448094 2026-08-17T11:28:44Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2144-2157].codfw.wmnet 2448095 wikitext text/x-wiki t3b9z7xj1lfmkahah3s06nlw4zb7fjf 2448096 2448095 2026-08-17T11:29:00Z Stashbot 7414 ladsgroup@deploy1003: Finished scap sync-world: Backport for [[gerrit:1326249|ImagePage: move TOC element below file link (T332644)]] (duration: 09m 56s) 2448096 wikitext text/x-wiki 74wrwhvzzert4jwbe9yur16ic13f7zd 2448097 2448096 2026-08-17T11:32:33Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.mysql.decommission 2448097 wikitext text/x-wiki cxrv96zkosjlg38cc4eqydd9g1gq9ej 2448098 2448097 2026-08-17T11:32:42Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.hosts.decommission for hosts db1153.eqiad.wmnet 2448098 wikitext text/x-wiki 77vhtm5dl2n392fb0lgjefn43mgfsme 2448099 2448098 2026-08-17T11:35:33Z Stashbot 7414 marostegui@dns1004: START - running authdns-update 2448099 wikitext text/x-wiki 6u1w8sqntyvs0ui457orvxexuxs9zlg 2448100 2448099 2026-08-17T11:36:21Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2144-2157].codfw.wmnet 2448100 wikitext text/x-wiki 0fa392tgpfjy1m6ipqf2vqqxabkx3vh 2448101 2448100 2026-08-17T11:37:27Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1058.eqiad.wmnet 2448101 wikitext text/x-wiki 0bktilld7qt7v6hsgatm3is7b1jybpm 2448102 2448101 2026-08-17T11:37:35Z Stashbot 7414 marostegui@dns1004: END - running authdns-update 2448102 wikitext text/x-wiki nzg4ryzuk2oc3ynuxvqe2vtgru43dfz 2448103 2448102 2026-08-17T11:38:06Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.dns.netbox 2448103 wikitext text/x-wiki jvd4llaly1ifr53tcn0qhrf7bre75nb 2448106 2448103 2026-08-17T11:38:50Z Stashbot 7414 jayme@deploy1003: helmfile [eqiad] DONE helmfile.d/admin 'sync'. 2448106 wikitext text/x-wiki rkn6qiio72d46348zc7332y03rxobr0 2448108 2448106 2026-08-17T11:41:14Z Stashbot 7414 fceratto@cumin1003: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host db2901.codfw.wmnet with OS trixie 2448108 wikitext text/x-wiki mvq0bpqzg4nxxygd6racl68wymv69d1 2448109 2448108 2026-08-17T11:41:17Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=97) for new host db2901.codfw.wmnet 2448109 wikitext text/x-wiki i1954mukiumm04lzb2byn4amjwflvmc 2448110 2448109 2026-08-17T11:41:58Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.ganeti.makevm for new host db1902.eqiad.wmnet 2448110 wikitext text/x-wiki 82l9oub020tdmxxlbrw7ro98qlul4q1 2448111 2448110 2026-08-17T11:42:32Z Stashbot 7414 marostegui@cumin1003: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db1153.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003" 2448111 wikitext text/x-wiki 6htnrbll4vvanr8p6hs28b1ti1ehrm9 2448112 2448111 2026-08-17T11:42:35Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1172: Pool db1172.eqiad.wmnet in after cloning 2448112 wikitext text/x-wiki i0ha6rkbwmdj30t653jybmuswidfjvv 2448113 2448112 2026-08-17T11:42:43Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1172.eqiad.wmnet onto db1286.eqiad.wmnet 2448113 wikitext text/x-wiki ru8t28ynr2jua7tueuqr0y6h169781v 2448114 2448113 2026-08-17T11:42:55Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: db1153.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003" 2448114 wikitext text/x-wiki jrvpg4cs0avhlgo5xnouz0chy5mm373 2448115 2448114 2026-08-17T11:42:57Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2448115 wikitext text/x-wiki jxuqphgh3jh1wgfdmzkt9mcieookwdh 2448116 2448115 2026-08-17T11:43:00Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts db1153.eqiad.wmnet 2448116 wikitext text/x-wiki k27hophtlta1tzhqz82vmeb52pl2o3b 2448117 2448116 2026-08-17T11:43:04Z Stashbot 7414 marostegui@cumin1003: Removing db1153 from zarcillo T434638 2448117 wikitext text/x-wiki 1jxk5v8ssb7aww7numwx65phlyegacn 2448118 2448117 2026-08-17T11:43:06Z Stashbot 7414 marostegui@cumin1003: END (PASS) - Cookbook sre.mysql.decommission (exit_code=0) 2448118 wikitext text/x-wiki qi50uhrzq17bysujamn863cyp0g8mwl 2448119 2448118 2026-08-17T11:43:16Z Stashbot 7414 fceratto@cumin1003: START - Cookbook sre.dns.netbox 2448119 wikitext text/x-wiki 5prdka9tupakwsfux9m0z8c7340l0eg 2448120 2448119 2026-08-17T11:43:29Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1058.eqiad.wmnet 2448120 wikitext text/x-wiki nh9bzhjyx28s6y3x3b46cyhox46n9lq 2448121 2448120 2026-08-17T11:43:34Z Stashbot 7414 jmm@cumin2003: END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1058.eqiad.wmnet 2448121 wikitext text/x-wiki 7wefzspjg1rwqfjak7z9ee4m0mkns1u 2448122 2448121 2026-08-17T11:44:06Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2144-2157].codfw.wmnet 2448122 wikitext text/x-wiki 0xyp9xvnpb4cleatdoe9qabipqw40tf 2448123 2448122 2026-08-17T11:44:09Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2144-2157].codfw.wmnet 2448123 wikitext text/x-wiki pjl71776sy3alh0vptgltdcyx7t0y68 2448124 2448123 2026-08-17T11:45:27Z Stashbot 7414 cgoubert@cumin2003: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2158-2171].codfw.wmnet 2448124 wikitext text/x-wiki 4vzh0q9planfse3u28u8q2fpr4z5vjm 2448125 2448124 2026-08-17T11:50:09Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1027.eqiad.wmnet 2448125 wikitext text/x-wiki jw1o9fts6mkfyshkn4gwt1mj0azluyj 2448126 2448125 2026-08-17T11:51:14Z Stashbot 7414 jayme: updated calico to v3.30.7 on wikikube eqiad - T427400 2448126 wikitext text/x-wiki 2tw6wqz3ujbfglen977x1yoqnrx0sjn 2448128 2448126 2026-08-17T11:52:59Z Stashbot 7414 cgoubert@cumin2003: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2158-2171].codfw.wmnet 2448128 wikitext text/x-wiki qt3tn96v1w27ntqtqctx02vif3oa3ln 2448129 2448128 2026-08-17T11:56:08Z Stashbot 7414 jmm@cumin2003: START - Cookbook sre.hosts.reboot-single for host ganeti1027.eqiad.wmnet 2448129 wikitext text/x-wiki kpsn866volti7r0j7jggd346mvmd6e5 2448130 2448129 2026-08-17T11:58:37Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) 2448130 wikitext text/x-wiki g694b3ww6gohjsfani45h88ffzq9zl2 2448131 2448130 2026-08-17T11:58:40Z Stashbot 7414 fceratto@cumin1003: END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host db1902.eqiad.wmnet 2448131 wikitext text/x-wiki fcbmobe8uquz2u57uc4pfeb0zifdpzh 498 55577 2447852 2446567 2026-08-17T07:24:23Z Stashbot 7414 wmbot~dcaro@acme: START - Cookbook wmcs.openstack.cloudvirt.vm_console 2447852 wikitext text/x-wiki oac80f2bplos1i0e3tplhg8loxmeeal 2447856 2447852 2026-08-17T07:25:25Z Stashbot 7414 wmbot~dcaro@acme: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255) 2447856 wikitext text/x-wiki gp91qr53aojhm671u7crhmwq4bykwzm 2447872 2447856 2026-08-17T07:33:29Z Stashbot 7414 dcaro: reboot node-2 2447872 wikitext text/x-wiki nakyp1m0nc0l4egcew8iew3tua0s0z4 498 443949 2447785 2444673 2026-08-16T18:09:09Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy a350d45 (Extract AnyOfPropertiesReferenceCheck from PropertyReferenceCheck) for T428637 2447785 wikitext text/x-wiki n8b4kow9dl8hln33l7mg7wqegozyzsj 2447786 2447785 2026-08-16T18:09:12Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy ad71cc9 (Simplify PropertyReferenceCheck to take a single property string) for T428637 2447786 wikitext text/x-wiki awvxz1zb8vqijztjs8rolim6peg1fov 2447787 2447786 2026-08-16T18:09:14Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 5c0ad71 (Introduce MultiPropertyReferenceCheck base class) for T428637 2447787 wikitext text/x-wiki s36k4artn4pi7x762labgw3zgntripp 2447788 2447787 2026-08-16T18:09:16Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy bc7a5b2 (Add AllPropertiesReferenceCheck with + syntax (AND)) for T428637 2447788 wikitext text/x-wiki pnk2uxyy1qujbddes0g56nx7mqj5lq1 2447789 2447788 2026-08-16T18:09:19Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy d1bba64 (Add P123/S248=Q19216625 syntax for value-constrained reference checks) for T428637 2447789 wikitext text/x-wiki jign714ln2zv8mbj3orjbkw63lj7z90 2447790 2447789 2026-08-16T18:09:21Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 8c5e68c (Refactor MultiPropertyReferenceCheck to use (property, value) tuples) for T428637 2447790 wikitext text/x-wiki jncao1hdjm16fum08v6cut309mnmh3u 2447791 2447790 2026-08-16T18:09:24Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy ba0e0f9 (Add value constraints in multi-property reference lists (S248=Q...+S813)) for T428637 2447791 wikitext text/x-wiki geb59x9qo7c6fm3maaeak981saq861k 2447792 2447791 2026-08-16T18:09:25Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 6c43294 (Refactor ColumnMaker.make() into focused helper methods) 2447792 wikitext text/x-wiki hwbgms7yy4b240kqr2bp9xzybzl5ims 2447793 2447792 2026-08-16T18:09:26Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 5bc0f3f (Extract _parse_bool_param helper for template boolean parsing) 2447793 wikitext text/x-wiki s5h1u4a34iqn8dmtn0xaye0v3sxa67e 2447794 2447793 2026-08-16T18:09:26Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 622e6ac (Rename stats_for_no_group → row_no_group) 2447794 wikitext text/x-wiki 6wjswik3sbyrcjz92pt95f7w3lqwihq 2447795 2447794 2026-08-16T18:09:28Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 2018d1e (Add row_totals template parameter to toggle totals row) 2447795 wikitext text/x-wiki 33urfsekark2s7dkhtnvysmpc2ux51b 2447796 2447795 2026-08-16T18:09:31Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy b4c3782 (Migrate deprecated template params during dashboard updates) 2447796 wikitext text/x-wiki lsei8sc13rsyfr9tk5sm8zbea1d1knn 2447801 2447796 2026-08-16T19:56:01Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy bd653aa (Remove Ansible playbook execution verbosity) 2447801 wikitext text/x-wiki nf7jga8rzkhhytttfeygsc1rs74e9ek 2447804 2447801 2026-08-16T22:42:32Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 2855e52 (Add pre-commit hook for Shell formatter shfmt) 2447804 wikitext text/x-wiki ogz18k7257zta0284l1qorzz7zsbimw 2447805 2447804 2026-08-16T22:42:33Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 6bd7972 (Reformat sal_messages_from_git_log.sh with shfmt) 2447805 wikitext text/x-wiki 14etbouk6s5coax69juxen7vct1adhd 2447806 2447805 2026-08-16T22:42:36Z Stashbot 7414 wmbot~jeanfred@tools-bastion-15: Deploy 269bec7 (Add shfmt reformat commit to .git-blame-ignore-revs) 2447806 wikitext text/x-wiki 3xlhgsysx0t922qu41kcpg95sulol30 0 449160 2447807 2447771 2026-08-17T00:02:48Z Dexbot 30554 Bot: Updating the report 2447807 wikitext text/x-wiki 2sveqdsuxv7bb3avagz86r76hfiz05x 0 453468 2447988 2443519 2026-08-17T09:53:18Z Tgr (WMF) 39134 /* Rotating */ 2447988 wikitext text/x-wiki atgp16mgu9tyme1buu6g4oalhqcpst0 116 453906 2447783 2447766 2026-08-16T15:42:30Z Gitlabaccountapprovalbot 37332 @pcendrer was approved. 2447783 wikitext text/x-wiki 6roj2l7b530fdhyu845yy1ueq4j3bwy 4 458615 2447808 2435590 2026-08-17T01:34:32Z Pine 3512 /* August 2026 Wikimedia Café meetups regarding Next 25 */ new section 2447808 wikitext text/x-wiki gmcyaen1u4pvceaxx07yfoopugaojcz 0 459156 2447958 2335838 2026-08-17T08:56:00Z Filippo Giunchedi 2939 /* Debugging */ 2447958 wikitext text/x-wiki 8opa3x66nvugf4zcmbdj5qx8y52diw5 0 459699 2447811 2447396 2026-08-17T02:57:43Z Stashbot 7414 eileen: civicrm upgraded from 105395c3 to 52efa44f 2447811 wikitext text/x-wiki 45fqbpn3nd3az7cke0xag7bykj49a3h 2447812 2447811 2026-08-17T03:06:00Z Stashbot 7414 eileen: civicrm upgraded from 52efa44f to ced9a157 2447812 wikitext text/x-wiki 8ccp7qy5o355dcx4t37tes0zknnm1vn 0 460375 2447781 2447779 2026-08-16T14:58:28Z AWesterinen-WMF 54189 /* POINT Decomposition */ 2447781 wikitext text/x-wiki ktuz1xjzijk5h3mu879c5ayu9e5i52d 2447782 2447781 2026-08-16T15:22:26Z AWesterinen-WMF 54189 /* geof:globe, geof:longitude and geof:latitude */ 2447782 wikitext text/x-wiki 7p17rsuyz7x15r10cs8f8b8tix84ys1 2447784 2447782 2026-08-16T15:51:25Z AWesterinen-WMF 54189 /* geof:globe, geof:longitude and geof:latitude */ 2447784 wikitext text/x-wiki 3lde2ba9mu8u9h5058kdw72ocrqfib8 2447797 2447784 2026-08-16T19:01:42Z AWesterinen-WMF 54189 /* geof:globe, geof:longitude and geof:latitude */ 2447797 wikitext text/x-wiki 8pl8dbv835yiac59is66ymzy4rtfi52 2447798 2447797 2026-08-16T19:02:11Z AWesterinen-WMF 54189 /* Limitations */ 2447798 wikitext text/x-wiki iiuvk1c609rbbbf8gjs4ct2uxicugt0 2447799 2447798 2026-08-16T19:09:14Z AWesterinen-WMF 54189 /* Limitations */ 2447799 wikitext text/x-wiki nwmqy04kubfavdtif9be8u6d4rw11ho 2447803 2447799 2026-08-16T20:21:37Z AWesterinen-WMF 54189 /* geof:globe, geof:longitude and geof:latitude */ 2447803 wikitext text/x-wiki 0ul0ovsujn89fbaah4cch3ukg0tfxq6 0 460534 2447817 2447558 2026-08-17T06:27:01Z SSapaty (WMF) 28955 incorporate various suggestions from the team 2447817 wikitext text/x-wiki nl9iqp5dck6sqc3e9sokxccyrtp6ryx