Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.45.0-wmf.8 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2320864 2320849 2025-07-06T16:28:24Z Stashbot 7414 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-75, tools-k8s-worker-nfs-8 2320864 wikitext text/x-wiki qj83zr0ikuc1as4xf7p88wnnqf91hys 2320865 2320864 2025-07-06T16:38:22Z Stashbot 7414 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-75, tools-k8s-worker-nfs-8 2320865 wikitext text/x-wiki d0nkthhs6bgyasg8npj3ymzq1v2tryh 2320895 2320865 2025-07-07T08:21:19Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logging 2320895 wikitext text/x-wiki 6cntedu106pxnnhsoikfw18p7ag2kmy 2320896 2320895 2025-07-07T08:26:34Z Stashbot 7414 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component logging 2320896 wikitext text/x-wiki jqk5xcqj4kp7rdwb3fr36j4fnooi1ad 2320926 2320896 2025-07-07T11:23:36Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld 2320926 wikitext text/x-wiki hfp3qs8r95kxv7v4l2fuswtwk3h61sa 2320927 2320926 2025-07-07T11:23:56Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld 2320927 wikitext text/x-wiki h6tq20wzequtwll419e591npzmkolsj 498 6726 2320893 2320631 2025-07-07T08:19:17Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component logging 2320893 wikitext text/x-wiki 2tfuxxmxvcjza7l8jbesszjwvzva9sj 2320894 2320893 2025-07-07T08:19:33Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component logging 2320894 wikitext text/x-wiki hv65gag538b4ppkaz9emuczr9w8h5wq 2320922 2320894 2025-07-07T11:11:34Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld 2320922 wikitext text/x-wiki q7usuzc0g30cmbamw6zu7f8laknucqd 2320923 2320922 2025-07-07T11:22:50Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld 2320923 wikitext text/x-wiki 1qtwjjr4whedj8fpqq389ioy0gckqj1 2320924 2320923 2025-07-07T11:23:01Z Stashbot 7414 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld 2320924 wikitext text/x-wiki 145i0mbnrpbki5sd8kfom532rventgj 2320925 2320924 2025-07-07T11:23:18Z Stashbot 7414 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld 2320925 wikitext text/x-wiki 10lm7ec5wta8ah8tf6fg3sr4de1k5ax 0 7919 2320868 2320842 2025-07-07T06:29:05Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply 2320868 wikitext text/x-wiki e42b5p8q7ug6mrnvpf0f6bg2ir4j5jm 2320870 2320868 2025-07-07T06:29:42Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply 2320870 wikitext text/x-wiki d8p3ectyayrjbhukb0vnwahrk3zke0b 2320874 2320870 2025-07-07T07:04:47Z Stashbot 7414 vgutierrez: testing haproxy 2.8.15 in cp5017 and cp5025 - T398720 2320874 wikitext text/x-wiki 2qctimndpd0f3k5tocnzl6fyn9bxh36 2320875 2320874 2025-07-07T07:10:07Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/services/mediawiki-dumps-legacy: apply 2320875 wikitext text/x-wiki 2g38emit0vfn35wotct8wolth5vjzk2 2320876 2320875 2025-07-07T07:11:05Z Stashbot 7414 brouberol@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/services/mediawiki-dumps-legacy: apply 2320876 wikitext text/x-wiki s10jbdm715mcmjqv2mht9r0lmc2kqw2 2320877 2320876 2025-07-07T07:13:57Z Stashbot 7414 marostegui@cumin1002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 15 hosts with reason: Primary switchover x1 T397612 2320877 wikitext text/x-wiki 1eo9zeozcs18w0ib4pu6932kg751slc 2320878 2320877 2025-07-07T07:22:00Z Stashbot 7414 marostegui@cumin1002: dbctl commit (dc=all): 'Set db1220 with weight 0 T397612', diff saved to https://phabricator.wikimedia.org/P78760 and previous config saved to /var/cache/conftool/dbconfig/20250707-072157-root.json 2320878 wikitext text/x-wiki r9hjclohusd5o9nwm0hzcifu0qpxap4 2320880 2320878 2025-07-07T07:25:22Z Stashbot 7414 marostegui: Starting x1 eqiad failover from db1237 to db1220 - T397612 2320880 wikitext text/x-wiki erf5zoegzo94iqeaytli0zszczri893 2320881 2320880 2025-07-07T07:25:46Z Stashbot 7414 vgutierrez: depooling cp7006 to test Ia82b9354a5b9e7bd5443b4af0888325919ddb19e - T397917 2320881 wikitext text/x-wiki b40z0fdil3khoohfduluial1sd4jsmr 2320882 2320881 2025-07-07T07:50:21Z Stashbot 7414 marostegui@dns1006: START - running authdns-update 2320882 wikitext text/x-wiki qlpi5l3c0i1axtuz0zhdu50jei1hyq1 2320883 2320882 2025-07-07T07:51:26Z Stashbot 7414 marostegui@dns1006: END - running authdns-update 2320883 wikitext text/x-wiki ginjni1acj8pyazsu5dx4zhseidgx0k 2320884 2320883 2025-07-07T07:52:58Z Stashbot 7414 marostegui@cumin1002: dbctl commit (dc=all): 'Promote db1220 to x1 primary and set section read-write T397612', diff saved to https://phabricator.wikimedia.org/P78762 and previous config saved to /var/cache/conftool/dbconfig/20250707-075254-root.json 2320884 wikitext text/x-wiki m8mgzhmceec1tzn9qpankpdotbc3esf 2320885 2320884 2025-07-07T07:53:10Z Stashbot 7414 marostegui@cumin1002: dbctl commit (dc=all): 'Depool db1237 T397612', diff saved to https://phabricator.wikimedia.org/P78763 and previous config saved to /var/cache/conftool/dbconfig/20250707-075308-root.json 2320885 wikitext text/x-wiki 3uerye6e8hada36imgnd1dvfv1a2wzu 2320886 2320885 2025-07-07T07:53:24Z Stashbot 7414 vgutierrez: repooling cp7006 with Ia82b9354a5b9e7bd5443b4af0888325919ddb19e applied - T397917 2320886 wikitext text/x-wiki 1kdvm1dn9pkdalee6qb7c5s7b5l9hma 2320887 2320886 2025-07-07T08:00:10Z Stashbot 7414 marostegui@cumin1002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1237.eqiad.wmnet with reason: Maintenance 2320887 wikitext text/x-wiki fl5ofzq40nscbp8qn10qo941vie13y4 2320888 2320887 2025-07-07T08:00:57Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "Feature: logging of deny actions; add rename functionality - oblivian@cumin1003" 2320888 wikitext text/x-wiki 7rsnf1vx8ouqwgwd8vcy0aaukyg01ii 2320889 2320888 2025-07-07T08:00:59Z Stashbot 7414 oblivian@cumin1003: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: Feature: logging of deny actions; add rename functionality - oblivian@cumin1003 2320889 wikitext text/x-wiki l44e9ty1ickj7ga4agmflwbvq6t9lo4 2320890 2320889 2025-07-07T08:01:30Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: Feature: logging of deny actions; add rename functionality - oblivian@cumin1003 2320890 wikitext text/x-wiki ejf2bd8uwf4t4xxi0kj0wvsf5hsnufu 2320891 2320890 2025-07-07T08:01:31Z Stashbot 7414 oblivian@cumin1003: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "Feature: logging of deny actions; add rename functionality - oblivian@cumin1003" 2320891 wikitext text/x-wiki qgegg8hzoykkv4266pw3naqnj9sr5q4 2320892 2320891 2025-07-07T08:15:39Z Stashbot 7414 marostegui@cumin1002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1237.eqiad.wmnet with reason: Maintenance 2320892 wikitext text/x-wiki gwnerdkz0ffhg5v1ae2eo32t62jp710 2320899 2320892 2025-07-07T09:06:42Z Stashbot 7414 marostegui@cumin1002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2160,2233].codfw.wmnet,db[1217,1228,1250].eqiad.wmnet with reason: maintenance 2320899 wikitext text/x-wiki rynjels51s5nk744guh5phcwxxubcoz 2320900 2320899 2025-07-07T09:09:14Z Stashbot 7414 elukey@cumin2002: START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS bookworm 2320900 wikitext text/x-wiki 69vd2eqqlp4jypc6qfx4acy5t4ikyg7 2320901 2320900 2025-07-07T09:13:54Z Stashbot 7414 marostegui: Failover m2 from db1250 to db1228 - T397633 2320901 wikitext text/x-wiki eil4r0o7d6x5moowl1c8p4vpp823oyq 2320902 2320901 2025-07-07T09:18:42Z Stashbot 7414 elukey@cumin2002: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS bookworm 2320902 wikitext text/x-wiki j7bhelmtlk5ddbpr5zc1ypuy5uoa6ga 2320903 2320902 2025-07-07T09:21:38Z Stashbot 7414 marostegui@cumin1002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1250.eqiad.wmnet with reason: Maintenance 2320903 wikitext text/x-wiki bct91mnk02ryfm7qgg17ex113w5cptf 2320904 2320903 2025-07-07T09:25:05Z Stashbot 7414 elukey@cumin2002: START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS bookworm 2320904 wikitext text/x-wiki oiglnxv8x5dm0l01esnr4aw9yqw07ws 2320905 2320904 2025-07-07T09:42:21Z Stashbot 7414 elukey@cumin2002: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS bookworm 2320905 wikitext text/x-wiki ov8tcp5a6ja6n090m318m85ehv0ax2i 2320906 2320905 2025-07-07T09:43:53Z Stashbot 7414 elukey@cumin2002: START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS bookworm 2320906 wikitext text/x-wiki 42is5wg2mcg4yv1i719n5aiwotwknx8 2320907 2320906 2025-07-07T09:58:10Z Stashbot 7414 elukey@cumin2002: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS bookworm 2320907 wikitext text/x-wiki 8m9klr90dnhz8fqn7y486xv76hn328c 2320908 2320907 2025-07-07T10:09:22Z Stashbot 7414 root@cumin1002: START - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies rolling restart_daemons on A:thanos-fe 2320908 wikitext text/x-wiki k0rk6xel07wpp3q2jp0rmjkfzos18l7 2320909 2320908 2025-07-07T10:13:17Z Stashbot 7414 root@cumin1002: END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies (exit_code=0) rolling restart_daemons on A:thanos-fe 2320909 wikitext text/x-wiki i2pk1zpfwj3dx1vfspb8b2b6bbu6ecz 2320910 2320909 2025-07-07T10:17:16Z Stashbot 7414 elukey@cumin2002: START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS bookworm 2320910 wikitext text/x-wiki q1lloefsdtzzg4057o24nxnh7zg5v6n 2320911 2320910 2025-07-07T10:24:09Z Stashbot 7414 Emperor: remove swift-account-stats_machinetranslation:prod time & service from thanos-fe1004 T335491 2320911 wikitext text/x-wiki 9okhri15yycb1rco7lvdvy562lwy43d 2320912 2320911 2025-07-07T10:30:58Z Stashbot 7414 elukey@cumin2002: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS bookworm 2320912 wikitext text/x-wiki pcrqyb242usahsbvwfy7t577oyzjtvz 2320913 2320912 2025-07-07T10:35:59Z Stashbot 7414 elukey@cumin2002: START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS bookworm 2320913 wikitext text/x-wiki ql6yk3w4g9kxk7772q887qlaol0qgwn 2320914 2320913 2025-07-07T10:45:00Z Stashbot 7414 marostegui@cumin1002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2160,2234].codfw.wmnet with reason: Maintenance 2320914 wikitext text/x-wiki la17rvkit5j8k5i3o8qu6wpow9azrxp 2320915 2320914 2025-07-07T10:45:16Z Stashbot 7414 elukey@cumin2002: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS bookworm 2320915 wikitext text/x-wiki otufpfy736uwwi74g80i3aiayqzkms2 2320916 2320915 2025-07-07T10:53:48Z Stashbot 7414 marostegui@cumin1002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1217.eqiad.wmnet with reason: Maintenance 2320916 wikitext text/x-wiki hipgosqqa2gtu3so33h998dzndm9ar0 2320917 2320916 2025-07-07T11:02:35Z Stashbot 7414 moritzm: installing modsecurity-apache security updates 2320917 wikitext text/x-wiki 87tvomm3sbiu3m8xo2i60voui56bucc 2320918 2320917 2025-07-07T11:06:00Z Stashbot 7414 marostegui@cumin1002: DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1250.eqiad.wmnet with reason: Maintenance 2320918 wikitext text/x-wiki g5qdtyqfbao02mzfi43r9lcvckjf9rm 2320919 2320918 2025-07-07T11:06:23Z Stashbot 7414 root@cumin1002: START - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies rolling restart_daemons on A:thanos-fe 2320919 wikitext text/x-wiki ptjsp1bnprjyawqonz64ecwdxjsmhmu 2320921 2320919 2025-07-07T11:10:17Z Stashbot 7414 root@cumin1002: END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-thanos-proxies (exit_code=0) rolling restart_daemons on A:thanos-fe 2320921 wikitext text/x-wiki 9mmeoy7oxqvlepbm27w9ulbbhv1aiis 2320928 2320921 2025-07-07T11:25:07Z Stashbot 7414 ladsgroup@deploy1003: Started scap sync-world: Backport for [[gerrit:1165169|Revert^2 "Clean up EventBus and jobs config"]] 2320928 wikitext text/x-wiki lqdzdhjj0cnknkujqrt46gckisifqsq 2320929 2320928 2025-07-07T11:42:20Z Stashbot 7414 elukey@cumin2002: START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS trixie 2320929 wikitext text/x-wiki 7flo6764iqax1k9rmrmbmenwm3hxg90 2320930 2320929 2025-07-07T11:46:22Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Backport for [[gerrit:1165169|Revert^2 "Clean up EventBus and jobs config"]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. 2320930 wikitext text/x-wiki 1dpn97qa9ggrkqymqyz71fbpkmv8rsx 2320931 2320930 2025-07-07T11:47:24Z Stashbot 7414 ladsgroup@deploy1003: ladsgroup: Continuing with sync 2320931 wikitext text/x-wiki 57q46ofjsmhow4tto366wxaots2al4d 2320932 2320931 2025-07-07T11:51:02Z Stashbot 7414 jmm@cumin2002: START - Cookbook sre.ganeti.changedisk for changing disk type of doh6001.wikimedia.org to drbd 2320932 wikitext text/x-wiki 1246bpt7ricnkjazhetggdfg2yo1ab6 2320933 2320932 2025-07-07T11:54:59Z Stashbot 7414 ladsgroup@cumin1002: dbctl commit (dc=all): 'Depool db2146 T398433', diff saved to https://phabricator.wikimedia.org/P78771 and previous config saved to /var/cache/conftool/dbconfig/20250707-115457-ladsgroup.json 2320933 wikitext text/x-wiki 3pifao4m378pm5bfq5hw3aaynixm150 2320934 2320933 2025-07-07T11:56:24Z Stashbot 7414 elukey@cumin2002: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS trixie 2320934 wikitext text/x-wiki fp2d1shwzkhxi2r2cb04hm6djvatnxm 0 17290 2320897 2320845 2025-07-07T08:45:26Z Stashbot 7414 hashar: gerrit: change mediawiki/* submit strategy to "Rebase if Necessary" and "Allow content merge" | T390719 2320897 wikitext text/x-wiki euw8qzbeumyrymae1r69gd8z7yz1pgf 2320898 2320897 2025-07-07T08:45:31Z Stashbot 7414 hashar: gerrit: change operations/* submit strategy to "Rebase if Necessary" and "Allow content merge" | T390719 2320898 wikitext text/x-wiki lhwg04gx8qy78wjye4z8ybpieop8pga 498 443800 2320873 2314860 2025-07-07T07:03:15Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-13: deployed 5b65c99d03 (upgrade dependencies, including mwparserfromhell 0.7.2) 2320873 wikitext text/x-wiki ooflckj507pk6nm97lyecuqfjh9ipeb 498 443946 2320872 2313354 2025-07-07T06:33:59Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-13: deployed cc28ff0494 (l10n updates: et, it, nn, pt-br, ru) 2320872 wikitext text/x-wiki t97e9jap0spv4y5xxntl0b0olwffzdb 498 443947 2320871 2311600 2025-07-07T06:31:20Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-13: deployed d1f4beb35e (l10n updates: af, ru, tr) 2320871 wikitext text/x-wiki d6d6bv5ijn13fnw4hxp1lhe0iixjk9t 498 447034 2320869 2311478 2025-07-07T06:29:18Z Stashbot 7414 wmbot~lucaswerkmeister@tools-bastion-13: deployed e059817c66 (l10n updates: fr, he, pt) 2320869 wikitext text/x-wiki 9zj6yti583vmxmm3qpmxbwj2elijz69 0 449160 2320867 2320859 2025-07-07T00:01:38Z Dexbot 30554 Bot: Updating the report 2320867 wikitext text/x-wiki 03nyz2vlalnladsv5r7hmy0b5fiw987 2320879 2320867 2025-07-07T07:23:29Z Dexbot 30554 Bot: Updating the report 2320879 wikitext text/x-wiki pimy2yvbexrza9inxzpdyr6ton9417n 2320935 2320879 2025-07-07T11:57:04Z Dexbot 30554 Bot: Updating the report 2320935 wikitext text/x-wiki hhlmvatobqpkkbm7w8ysklzzwsgxw6m 3 454542 2320863 2271090 2025-07-06T13:50:03Z XXBlackburnXx 38000 XXBlackburnXx moved page [[User talk:NNNH]] to [[User talk:Renamed user c94112c361f80e601af4d9516826efa3]] without leaving a redirect: Automatically moved page while renaming the user "[[Special:CentralAuth/NNNH|NNNH]]" to "[[Special:CentralAuth/Renamed user c94112c361f80e601af4d9516826efa3|Renamed user c94112c361f80e601af4d9516826efa3]]" 2271090 wikitext text/x-wiki 0wjrh0br4u4s5v9psz2p1thuh1vey1t 3 457329 2320866 2270037 2025-07-06T20:34:33Z J0rd1 37490 Blanked the page 2320866 wikitext text/x-wiki phoiac9h4m842xq45sp7s6u21eteeq1 3 459034 2320920 2025-07-07T11:06:51Z StrikerBot 8475 Welcome to Toolforge! 2320920 wikitext text/x-wiki bbi0bsu6mn94gmtxu34m15ue7sxgdam